icinga2/tools/migration/icinga2-migrate-config

4695 lines
210 KiB
Plaintext
Raw Normal View History

#!/usr/bin/env perl
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
=pod
/******************************************************************************
* Icinga 2 *
2013-09-25 07:43:57 +02:00
* Copyright (C) 2012-2013 Icinga Development Team (http://www.icinga.org/) *
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
* *
* This program is free software; you can redistribute it and/or *
* modify it under the terms of the GNU General Public License *
* as published by the Free Software Foundation; either version 2 *
* of the License, or (at your option) any later version. *
* *
* This program is distributed in the hope that it will be useful, *
* but WITHOUT ANY WARRANTY; without even the implied warranty of *
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the *
* GNU General Public License for more details. *
* *
* You should have received a copy of the GNU General Public License *
* along with this program; if not, write to the Free Software Foundation *
* Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301, USA. *
******************************************************************************/
=cut
=head1 NAME
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
icinga2-migrate-config - Migrate Icinga 1.x config to Icinga 2 format
=head1 SYNOPSIS
icinga2-migrate-config -c <path to icinga.cfg from Icinga 1.x>
-o <output directory for Icinga 2 config>
[-O]
[-v]
[-h]
[-V]
Migrate Icinga 1.x configuration to new Icinga 2 configuration format.
=head1 OPTIONS
=over
=item -c|--icingacfgfile <path to icinga.cfg>
Path to the Icinga 1.x main configuration file "icinga.cfg".
=item -o|--outputcfgdir <output directory for icinga2 config>
Directory to the Icinga 2 configuration output.
=item -O|--objectscache
Use objects cache file instead of cfg_dir|file in icinga.cfg.
=item --hosttmpl=<template name>
Custom template name for all host objects.
=item --servicetmpl=<template name>
Custom template name for all service objects.
=item --usertmpl=<template name>
Custom template name for all user objects.
=item --timeperiodtmpl=<template name>
Custom template name for all timeperiod objects.
=item --notificationtmpl=<template name>
Custom template name for all notification objects.
=item --checkcommandtmpl=<template name>
Custom template name for all checkcommand objects.
=item --notificationcommandtmpl=<template name>
Custom template name for all notificationcommand objects.
=item --eventcommandtmpl=<template name>
Custom template name for all eventcommand objects.
=item -v|--verbose
Verbose mode.
=item -h|--help
Print help page.
=item -V|--version
print version.
=cut
#use warnings;
#use strict;
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
use Data::Dumper;
use File::Find;
use Storable qw(dclone);
use Getopt::Long qw(:config no_ignore_case bundling);
use Pod::Usage;
use POSIX qw(strftime);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
use feature 'say';
my $version = "0.0.3";
# get command-line parameters
our $opt;
GetOptions(
"c|icingacfgfile=s" => \$opt->{icinga1xcfg},
"o|outputcfgdir=s" => \$opt->{icinga2xoutputprefix},
"O|objectscache" => \$opt->{icinga1xobjectscache},
"hosttmpl=s" => \$opt->{hosttmpl},
"servicetmpl=s" => \$opt->{servicetmpl},
"usertmpl=s" => \$opt->{usertmpl},
"notificationtmpl=s" => \$opt->{notificationtmpl},
"timeperiodtmpl=s" => \$opt->{timeperiodtmpl},
"checkcommandtmpl=s" => \$opt->{checkcommandtmpl},
"notificationcommandtmpl=s" => \$opt->{notificationcommandtmpl},
"eventcommandtmpl=s" => \$opt->{eventcommandtmpl},
"tmpl=s" => \$opt->{tmpl},
"tmpl=s" => \$opt->{tmpl},
"v|verbose" => \$opt->{verbose},
"h|help" => \$opt->{help},
"V|version" => \$opt->{version}
);
my $icinga1_cfg = "";
my $icinga2_cfg = {};
my $conf_prefix = "./conf";
my $verbose = 0;
our $dbg_lvl = 0;
$icinga2_cfg->{'__I2EXPORT_DEBUG'} = 0;
if(defined($opt->{icinga1xcfg})) {
$icinga1_cfg = $opt->{icinga1xcfg};
}
if(defined($opt->{icinga2xoutputprefix})) {
$conf_prefix = $opt->{icinga2xoutputprefix};
}
if(defined($opt->{verbose})) {
$verbose = $opt->{verbose};
$icinga2_cfg->{'__I2EXPORT_DEBUG'} = 1;
$dbg_lvl = 1;
}
if (defined $opt->{version}) { print $version."\n"; exit 0; }
if ($opt->{help}) { pod2usage(1); }
$icinga2_cfg->{'main'}= "$conf_prefix/icinga2.conf";
$icinga2_cfg->{'resource'}= "$conf_prefix/resource.conf";
$icinga2_cfg->{'hosts'}= "$conf_prefix/hosts.conf";
$icinga2_cfg->{'services'}= "$conf_prefix/services.conf";
$icinga2_cfg->{'users'}= "$conf_prefix/users.conf";
$icinga2_cfg->{'groups'}= "$conf_prefix/groups.conf";
$icinga2_cfg->{'notifications'}= "$conf_prefix/notifications.conf";
$icinga2_cfg->{'timeperiods'}= "$conf_prefix/timeperiods.conf";
$icinga2_cfg->{'commands'}= "$conf_prefix/commands.conf";
$icinga2_cfg->{'itl'}->{'host-template'} = "";
$icinga2_cfg->{'itl'}->{'service-template'} = "";
$icinga2_cfg->{'itl'}->{'user-template'} = "";
$icinga2_cfg->{'itl'}->{'notification-template'} = "";
$icinga2_cfg->{'itl'}->{'timeperiod-template'} = "legacy-timeperiod";
$icinga2_cfg->{'itl'}->{'checkcommand-template'} = "plugin-check-command";
$icinga2_cfg->{'itl'}->{'notificationcommand-template'} = "plugin-notification-command";
$icinga2_cfg->{'itl'}->{'eventcommand-template'} = "plugin-event-command";
if (defined($opt->{hosttmpl})) {
$icinga2_cfg->{'customtmpl'}->{'host-template'} = $opt->{hosttmpl};
print "Custom host template: '" . $icinga2_cfg->{'customtmpl'}->{'host-template'} . "'\n";
}
if (defined($opt->{servicetmpl})) {
$icinga2_cfg->{'customtmpl'}->{'service-template'} = $opt->{servicetmpl};
print "Custom service template: '" . $icinga2_cfg->{'customtmpl'}->{'service-template'} . "'\n";
}
if (defined($opt->{usertmpl})) {
$icinga2_cfg->{'customtmpl'}->{'user-template'} = $opt->{usertmpl};
print "Custom user template: '" . $icinga2_cfg->{'customtmpl'}->{'user-template'} . "'\n";
}
if (defined($opt->{notificationtmpl})) {
$icinga2_cfg->{'customtmpl'}->{'notification-template'} = $opt->{notificationtmpl};
print "Custom notification template: '" . $icinga2_cfg->{'customtmpl'}->{'notification-template'} . "'\n";
}
if (defined($opt->{timeperiodtmpl})) {
$icinga2_cfg->{'customtmpl'}->{'timeperiod-template'} = $opt->{timeperiodtmpl};
print "Custom timeperiod template: '" . $icinga2_cfg->{'customtmpl'}->{'timeperiod-template'} . "'\n";
}
if (defined($opt->{checkcommandtmpl})) {
$icinga2_cfg->{'customtmpl'}->{'checkcommand-template'} = $opt->{checkcommandtmpl};
print "Custom checkcommand template: '" . $icinga2_cfg->{'customtmpl'}->{'checkcommand-template'} . "'\n";
}
if (defined($opt->{notificationcommandtmpl})) {
$icinga2_cfg->{'customtmpl'}->{'notificationcommand-template'} = $opt->{notificationcommandtmpl};
print "Custom notificationcommand template: '" . $icinga2_cfg->{'customtmpl'}->{'notificationcommand-template'} . "'\n";
}
if (defined($opt->{eventcommandtmpl})) {
$icinga2_cfg->{'customtmpl'}->{'eventcommand-template'} = $opt->{eventcommandtmpl};
print "Custom eventcommand template: '" . $icinga2_cfg->{'customtmpl'}->{'eventcommand-template'} . "'\n";
}
my $type_cnt;
################################################################################
# MAIN
################################################################################
# verify that we have something to read
if ( !-f $icinga1_cfg) {
print "ERROR: Icinga 1.x config file '$icinga1_cfg' not found.\n";
pod2usage(1);
}
# the import
debug("Start Icinga 1.x config import @".time);
my $icinga1_cfg_obj;
if (!defined($opt->{icinga1xobjectscache})) {
$icinga1_cfg_obj = parse_icinga1_objects($icinga1_cfg);
} else {
$icinga1_cfg_obj = parse_icinga1_objects_cache($icinga1_cfg);
}
# global macros
my $icinga1_global_macros = parse_icinga1_global_macros($icinga1_cfg);
# the conversion magic inside
debug("Start Icinga 1.x config migration @".time);
my $icinga2_cfg_obj = migrate_2x($icinga2_cfg, $icinga1_cfg_obj, $icinga1_cfg_obj_cache, $icinga1_global_macros);
# the export
debug("Dump Icinga 2.x config @".time);
dump_cfg_resource_2x($icinga2_cfg, $icinga1_global_macros);
dump_cfg_obj_2x($icinga2_cfg, $icinga2_cfg_obj);
# print stats
print "######################################################################\n";
print "# CONVERSION STATISTICS\n";
print "######################################################################\n";
print_sorted_hash($icinga2_cfg_obj->{'__I2_CONVERT_STATS'});
debug("Icinga 2.x config migratione done @".time);
=pod
IMPORT OPTIONS
=cut
################################################################################
# PARSE 1.x
################################################################################
sub get_key_from_icinga1_main_cfg {
my ($file, $key) = @_;
my @key_arr = ();
if ( !-f $file) {
print "cfg file $file does not exist!";
return;
}
if ( open ( my $fh, '<', $file ) ) {
while ( my $line = <$fh> ) {
chomp($line);
$line =~ s/#.*//;
if ($line =~ /^\s*$key=([^\s]+)/) {
push @key_arr, $1; # we may have multiple occurences
}
}
}
return @key_arr;
}
sub parse_icinga1_resource_cfg {
my $file = shift;
my @cfg = slurp($file);
my $global_macros = {};
foreach my $line (@cfg) {
$line = strip($line);
# skip comments and empty lines
next if ($line eq "" || !defined($line) || $line =~ /^\s+$/);
next if ($line =~ /^[#;]/ || $line =~ /;.*/);
#debug($line);
my ($macro_name, $macro_value) = split /=/, $line, 2;
$macro_name =~ /\$(.*)\$/;
$macro_name = $1;
$global_macros->{$macro_name} = escape_str($macro_value);
}
return $global_macros;
}
sub parse_icinga1_global_macros {
my $icinga1_cfg = shift;
my $global_macros;
my ($icinga1_resource_file) = get_key_from_icinga1_main_cfg($icinga1_cfg, "resource_file");
# resource.cfg
if(defined($icinga1_resource_file)) {
$global_macros = parse_icinga1_resource_cfg($icinga1_resource_file);
}
# special attributes in icinga.cfg (admin_*)
my ($admin_pager) = get_key_from_icinga1_main_cfg($icinga1_cfg, "admin_pager");
my ($admin_email) = get_key_from_icinga1_main_cfg($icinga1_cfg, "admin_email");
if (defined($admin_pager)) {
$global_macros->{'ADMINPAGER'} = $admin_pager;
}
if(defined($admin_email)) {
$global_macros->{'ADMINEMAIL'} = $admin_email;
}
return $global_macros;
}
sub parse_icinga1_object_cfg {
my $cfg_obj = shift;
my $file = shift;
my $obj = {}; #hashref
my $in_define = 0;
my $in_timeperiod = 0;
my $type;
my $append; # this is a special case where multiple lines are appended with \ - not sure if we support THAT.
my $inline_comment;
my $attr;
my $val;
my @cfg = slurp($file);
#debug("========================================================");
#debug("File: $file");
foreach my $line (@cfg) {
$line = strip($line);
#debug("Processing line: '$line'");
# skip comments and empty lines
next if ($line eq "" || !defined($line) || $line =~ /^\s+$/);
next if ($line =~ /^[#;]/);
# || $line =~ /;.*/);
$line =~ s/[\r\n\s]+$//;
$line =~ s/^\s+//;
# end of def
if ($line =~ /}(\s*)$/) {
$in_define = undef;
# store type for later
$cfg_obj->{'type_cnt'}->{$type} = $cfg_obj->{'type_cnt'}->{$type} + 1;
$type = "";
next;
}
# start of def
elsif ($line =~ /define\s+(\w+)\s*{?(.*)$/) {
$type = $1;
$append = $2;
next if (!defined($type));
if ($type eq "timeperiod") {
$in_timeperiod = 1;
} else {
$in_timeperiod = 0;
}
# save the type
$cfg_obj->{$type}->{$cfg_obj->{'type_cnt'}->{$type}}->{'__I2CONVERT_TYPE'} = $type;
# we're ready to process entries
$in_define = 1;
# save the current type counter, being our unique key here
next;
}
# in def
elsif ($in_define == 1) {
# first, remove the annoying inline comments after ';'
$line =~ s/\s*[;\#](.*)$//;
$inline_comment = $1;
# then split it and save it by type->cnt->attr->val
#($attr, $val) = split (/\s+/, $line, 2); # important - only split into 2 elements
# timeperiods require special parser
if ($in_timeperiod == 1) {
if ($line =~ /timeperiod_name/ || $line =~ /alias/ || $line =~ /exclude/) {
$line =~ m/([\w]+)\s*(.*)/;
$attr = strip($1); $val = strip($2);
} else {
$line =~ m/(.*)\s+([\d\W]+)/;
$attr = strip($1); $val = strip($2);
}
} else {
$line =~ m/([\w]+)\s*(.*)/;
$attr = strip($1); $val = strip($2);
}
# ignore empty values
next if (!defined($val));
next if ($val eq "");
#debug("cnt: $cfg_obj->{'type_cnt'}->{$type}");
#debug("line: '$line'");
#debug("type: $type");
#debug("attr: $attr");
#debug("val: $val");
#debug("\n");
# strip illegal object name characters, replace with _
if ( ($attr =~ /name/ && $attr !~ /display_name/) ||
$attr =~ /description/ ||
$attr =~ /contact/ ||
$attr =~ /groups/ ||
$attr =~ /members/ ||
$attr =~ /use/ ||
$attr =~ /parents/ ||
$attr =~ /_period/
) {
$val = strip_object_name($val);
}
# treat 'null' (disable) as '0'
if ($val eq "null") {
$val = 0;
}
$cfg_obj->{$type}->{$cfg_obj->{'type_cnt'}->{$type}}->{$attr} = $val;
# ignore duplicated attributes, last one wins
}
else {
$in_define = 0;
}
}
#debug("========================================================");
return $cfg_obj;
}
# the idea is to reduce work load - get all the existing object relations (host->service)
# and have core 1.x already mapped that. we focus on getting the details when
# needed, but do not print the object without templates - only if there's no other way.
sub parse_icinga1_objects_cache {
my $icinga1_cfg = shift;
# functions return array in case of multiple occurences, we'll take only the first one
my ($object_cache_file) = get_key_from_icinga1_main_cfg($icinga1_cfg, "object_cache_file");
if(!defined($object_cache_file)) {
print "ERROR: No objects cache file found in $icinga1_cfg! We'll need for final object conversion.\n";
return -1;
}
if(! -r $object_cache_file) {
print "ERROR: objects cache file '$object_cache_file' from $icinga1_cfg not found! We'll need it for final object conversion.\n";
return -1;
}
my $cfg_obj_cache = {};
print "Processing file '$object_cache_file'...\n";
$cfg_obj_cache = parse_icinga1_object_cfg($cfg_obj_cache, $object_cache_file);
#say Dumper($cfg_obj_cache);
return $cfg_obj_cache;
}
# parse all existing config object included in icinga.cfg, with all their templates
# and grouping tricks
sub parse_icinga1_objects {
my $icinga1_cfg = shift;
my @cfg_files = get_key_from_icinga1_main_cfg($icinga1_cfg, "cfg_file");
my @cfg_dirs = get_key_from_icinga1_main_cfg($icinga1_cfg, "cfg_dir");
sub find_icinga1_cfg_files {
my $file = $File::Find::name;
return if -d $file;
if ($file =~ /\.cfg$/) {
push @cfg_files, $file;
}
}
foreach my $cfg_dir (@cfg_dirs) {
find(\&find_icinga1_cfg_files, $cfg_dir);
}
# check if there was nothing to include
if (!@cfg_files) {
print "ERROR: $icinga1_cfg did not contain any object includes.\n";
return -1;
}
#print "@cfg_files";
# now fetch all the config information into our global hash ref
my $cfg_objs = {};
foreach my $cfg_file (@cfg_files) {
print "Processing file '$cfg_file'...\n";
$cfg_objs = parse_icinga1_object_cfg($cfg_objs, $cfg_file);
}
#say Dumper($cfg_obj);
#say Dumper($cfg_obj->{'service'});
return $cfg_objs;
}
=pod
MIGRATION
=cut
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
################################################################################
## Validation
#################################################################################
sub obj_1x_is_template {
my $obj_1x = shift;
if (defined($obj_1x->{'register'})) {
if ($obj_1x->{'register'} == 0) {
return 1;
}
}
return 0;
}
sub obj_1x_uses_template {
my $obj_1x = shift;
if (defined($obj_1x->{'use'})) {
return 1;
}
return 0;
}
# check if notification object exists (2.x only)
sub obj_2x_notification_exists {
my $objs = shift;
my $obj_type = 'notification';
my $obj_attr = '__I2CONVERT_NOTIFICATION_OBJECT_NAME'; # this must be set outside, no matter if template or not XXX
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
my $obj_val = shift;
#debug("My objects hive: ".Dumper($objs));
#debug("Checking for type=$obj_type attr=$obj_attr val=$obj_val ");
foreach my $obj_key (keys %{@$objs{$obj_type}}) {
my $obj = @$objs{$obj_type}->{$obj_key};
next if !defined($obj->{$obj_attr});
#debug("Getting attr $obj_attr and val $obj_val");
if ($obj->{$obj_attr} eq $obj_val) {
#debug("Found object: ".Dumper($obj));
return 1;
}
}
return 0;
}
# check if command object exists (2.x only)
sub obj_2x_command_exists {
my $objs = shift;
my $obj_type = 'command';
my $obj_attr = '__I2CONVERT_COMMAND_NAME';
my $obj_val = shift;
my $command_type = shift;
my $command_type_attr = '__I2CONVERT_COMMAND_TYPE';
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
#debug("My objects hive: ".Dumper($objs));
#debug("Checking for type=$obj_type attr=$obj_attr val=$obj_val ");
foreach my $obj_key (keys %{@$objs{$obj_type}}) {
my $obj = @$objs{$obj_type}->{$obj_key};
next if !defined($obj->{$obj_attr});
#debug("Getting attr $obj_attr and val $obj_val");
next if $obj->{$command_type_attr} ne $command_type;
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
if ($obj->{$obj_attr} eq $obj_val) {
#debug("Found object: ".Dumper($obj));
return 1;
}
}
return 0;
}
# check if host service object exists (2.x only)
sub obj_2x_host_service_exists {
my $objs = shift;
my $obj_attr = '__I2CONVERT_SERVICEDESCRIPTION';
my $obj_val = shift;
#debug("My objects hive: ".Dumper($objs));
#debug("Checking for type=$obj_type attr=$obj_attr val=$obj_val ");
foreach my $obj_key (keys %{$objs}) {
my $obj = $objs->{$obj_key};
next if !defined($obj->{$obj_attr});
#debug("Getting attr $obj_attr and val $obj_val");
if ($obj->{$obj_attr} eq $obj_val) {
#debug("Found object: ".Dumper($obj));
#say Dumper($obj);
return 1;
}
}
return 0;
}
# check if hostcheck service template object exists (2.x only)
sub obj_2x_hostcheck_service_template_exists {
my $objs = shift;
my $obj_attr = '__I2CONVERT_TEMPLATE_NAME';
my $obj_val = shift;
#debug("My objects hive: ".Dumper($objs));
#debug("Checking for type=$obj_type attr=$obj_attr val=$obj_val ");
foreach my $obj_key (keys %{$objs}) {
my $obj = $objs->{$obj_key};
next if !defined($obj->{$obj_attr});
#debug("Getting attr $obj_attr and val $obj_val");
if ($obj->{$obj_attr} eq $obj_val) {
#debug("Found object: ".Dumper($obj));
#say Dumper($obj);
return 1;
}
}
return 0;
}
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
#################################################################################
# Migration
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
#################################################################################
#################################################################################
# Get Object Helpers
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
#################################################################################
# get host object by attr 'host_name'
sub obj_get_host_obj_by_host_name {
my $objs = shift;
my $obj_type = 'host';
my $obj_attr = 'host_name';
my $obj_val = shift;
#debug("My objects hive: ".Dumper($objs));
#debug("Checking for type=$obj_type attr=$obj_attr val=$obj_val ");
foreach my $obj_key (keys %{@$objs{$obj_type}}) {
my $obj = @$objs{$obj_type}->{$obj_key};
next if !defined($obj->{$obj_attr});
#debug("Getting attr $obj_attr and val $obj_val");
if ($obj->{$obj_attr} eq $obj_val) {
#debug("Found object: ".Dumper($obj));
return $obj;
}
}
return undef;
}
# get service object by attr 'host_name' and 'service_description'
sub obj_get_service_obj_by_host_name_service_description {
my $objs = shift;
my $obj_type = 'service';
my $obj_attr_host = shift;
my $obj_attr_service = shift;
my $obj_val_host = shift;
my $obj_val_service = shift;
#debug("My objects hive: ".Dumper($objs));
#debug("Checking for service with host_name=$obj_val_host service_description=$obj_val_service ");
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
foreach my $obj_key (keys %{@$objs{$obj_type}}) {
my $obj = @$objs{$obj_type}->{$obj_key};
next if !defined($obj->{$obj_attr_host});
next if !defined($obj->{$obj_attr_service});
#debug("Getting attr $obj_attr_host/$obj_attr_service and val $obj_val_host/$obj_val_service");
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
if (($obj->{$obj_attr_host} eq $obj_val_host) && ($obj->{$obj_attr_service} eq $obj_val_service)) {
#debug("Found object: ".Dumper($obj));
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
return $obj;
}
}
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
return undef;
}
# get contact object by attr 'contact_name'
sub obj_get_contact_obj_by_contact_name {
my $objs = shift;
my $obj_type = shift;
my $obj_attr = shift;
my $obj_val = shift;
#debug("My objects hive: ".Dumper($objs));
#debug("Checking for type=$obj_type attr=$obj_attr val=$obj_val ");
foreach my $obj_key (keys %{@$objs{$obj_type}}) {
my $obj = @$objs{$obj_type}->{$obj_key};
next if !defined($obj->{$obj_attr});
#debug("Getting attr $obj_attr and val $obj_val");
if ($obj->{$obj_attr} eq $obj_val) {
#debug("Found object: ".Dumper($obj));
return $obj;
}
}
return undef;
}
# get user object by attr 'user_name'
sub obj_get_user_obj_by_user_name {
my $objs = shift;
my $obj_type = 'user';
my $obj_attr = 'user_name';
my $obj_val = shift;
#debug("My objects hive: ".Dumper($objs));
#debug("Checking for type=$obj_type attr=$obj_attr val=$obj_val ");
foreach my $obj_key (keys %{@$objs{$obj_type}}) {
my $obj = @$objs{$obj_type}->{$obj_key};
next if !defined($obj->{$obj_attr});
#debug("Getting attr $obj_attr and val $obj_val");
if ($obj->{$obj_attr} eq $obj_val) {
#debug("Found object: ".Dumper($obj));
return $obj;
}
}
return undef;
}
# get template object by attr 'name'
sub obj_get_tmpl_obj_by_tmpl_name {
my $objs = shift;
my $obj_tmpl_type = shift;
my $obj_attr_tmpl_name = shift;
my $obj_attr = 'name';
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
#debug("My objects hive: ".Dumper($objs));
#debug("Checking for template name with $obj_attr_tmpl_name");
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
foreach my $obj_key (keys %{@$objs{$obj_tmpl_type}}) {
my $obj = @$objs{$obj_tmpl_type}->{$obj_key};
next if !defined($obj->{$obj_attr});
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# XXX it would be safe, but we cannot garantuee it here, so better check before if we want a template or not
if ($obj->{$obj_attr} eq $obj_attr_tmpl_name) {
#debug("Found object: ".Dumper($obj));
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
return $obj;
}
}
return undef;
}
# get hostgroup object by attr 'hostgroup_name'
sub obj_get_hostgroup_obj_by_hostgroup_name {
my $objs = shift;
my $obj_type = 'hostgroup';
my $obj_attr = 'hostgroup_name';
my $obj_val = shift;
#debug("My objects hive: ".Dumper($objs));
#debug("Checking for type=$obj_type attr=$obj_attr val=$obj_val ");
foreach my $obj_key (keys %{@$objs{$obj_type}}) {
my $obj = @$objs{$obj_type}->{$obj_key};
next if !defined($obj->{$obj_attr});
#debug("Getting attr $obj_attr and val $obj_val");
if ($obj->{$obj_attr} eq $obj_val) {
#debug("Found object: ".Dumper($obj));
return $obj;
}
}
return undef;
}
#################################################################################
# Get Object Attribute Helpers
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
#################################################################################
# get host_names by attr 'hostgroup_name'
sub obj_get_hostnames_arr_by_hostgroup_name {
my $objs = shift;
my $obj_type = 'host';
my $obj_attr = 'hostgroups';
my $obj_val = shift;
my @host_names = ();
#debug("My objects hive: ".Dumper($objs));
#debug("Checking for type=$obj_type attr=$obj_attr val=$obj_val ");
foreach my $obj_key (keys %{@$objs{$obj_type}}) {
my $obj = @$objs{$obj_type}->{$obj_key};
next if !defined($obj->{$obj_attr});
#debug("Getting attr $obj_attr and val $obj_val");
foreach my $hg (@{$obj->{$obj_attr}}) {
if ($hg eq $obj_val) {
#debug("Found object: ".Dumper($obj));
push @host_names, $obj->{'host_name'};
}
}
}
return @host_names;
}
sub obj_get_usernames_arr_by_usergroup_name {
my $objs = shift;
my $obj_type = 'user';
my $obj_attr = 'usergroups';
my $obj_val = shift;
my @user_names = ();
#debug("My objects hive: ".Dumper($objs));
#debug("Checking for type=$obj_type attr=$obj_attr val=$obj_val ");
foreach my $obj_key (keys %{@$objs{$obj_type}}) {
my $obj = @$objs{$obj_type}->{$obj_key};
next if !defined($obj->{$obj_attr});
#debug("Getting attr $obj_attr and val $obj_val");
foreach my $user (@{$obj->{$obj_attr}}) {
if ($user eq $obj_val) {
#debug("Found object: ".Dumper($obj));
push @user_names, $obj->{'user_name'};
}
}
}
return @user_names;
}
# used after relinking all services with servicegroups
sub obj_2x_get_service_arr_by_servicegroup_name {
my $objs_2x = shift;
my $objs = $objs_2x;
my $obj_type = 'service';
my $obj_attr = 'servicegroups';
my $obj_val = shift;
my @service_names = ();
#debug("My objects hive: ".Dumper($objs));
#debug("Checking for type=$obj_type attr=$obj_attr val=$obj_val ");
foreach my $obj_key (keys %{@$objs{$obj_type}}) {
my $obj = @$objs{$obj_type}->{$obj_key};
# this there's no attr, try template tree
my @servicegroups = ();
my $host_name = $obj->{'__I2CONVERT_SERVICE_HOSTNAME'};
# skip invalid resolved objects
if (!defined($host_name)) {
#say Dumper("missing host name...");
next;
}
if (defined($obj->{$obj_attr}) && scalar(@{$obj->{$obj_attr}} > 0)) {
push @servicegroups, @{$obj->{$obj_attr}};
#say Dumper("$obj_attr ========== found in object $obj->{'__I2CONVERT_SERVICE_HOSTNAME'}:$obj->{'__I2CONVERT_SERVICEDESCRIPTION'}");
#say Dumper(@servicegroups);
} else {
#say Dumper("START ------------------------");
#say Dumper($obj);
my @service_sgs = obj_2x_get_service_servicegroups($objs_2x,$obj,$host_name,$obj_attr);
#say Dumper(@service_sgs);
if (scalar @service_sgs > 0) {
push @servicegroups, @service_sgs;
#say Dumper("$obj_attr ========== found in template tree $obj->{'__I2CONVERT_SERVICE_HOSTNAME'}:$obj->{'__I2CONVERT_SERVICEDESCRIPTION'}");
#say Dumper(@servicegroups);
#say Dumper($obj);
}
#say Dumper("END ------------------------");
}
#debug("Getting attr $obj_attr and val $obj_val");
# check if servicegroup_name is in the array of servicegroups for this processed service
foreach my $servicegroup (@servicegroups) {
# skip templates
next if ($obj->{'__I2CONVERT_SERVICE_IS_TEMPLATE'} == 1);
if ($servicegroup eq $obj_val) {
#debug("Found object: ".Dumper($obj));
my $service_name;
$service_name->{'__I2CONVERT_SERVICE_HOSTNAME'} = $obj->{'__I2CONVERT_SERVICE_HOSTNAME'};
$service_name->{'__I2CONVERT_SERVICEDESCRIPTION'} = $obj->{'__I2CONVERT_SERVICEDESCRIPTION'};
push @service_names, $service_name;
}
}
}
return @service_names;
}
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
sub obj_1x_get_all_hostnames_arr {
my $objs = shift;
my $obj_type = 'host';
my $obj_attr = 'host_name';
my $obj_val = '*';
my @host_names = ();
#debug("My objects hive: ".Dumper($objs));
#debug("Checking for type=$obj_type attr=$obj_attr val=$obj_val ");
foreach my $obj_key (keys %{@$objs{$obj_type}}) {
my $obj = @$objs{$obj_type}->{$obj_key};
next if !defined($obj->{$obj_attr});
#debug("Getting attr $obj_attr and val $obj_val");
push @host_names, $obj->{$obj_attr};
}
return @host_names;
}
# get host_name from object
sub obj_1x_get_host_host_name {
my $objs_1x = shift;
my $obj_1x = shift;
my $host_name = "";
# if this object is invalid, bail early
return undef if !defined($obj_1x);
# first, check if we already got a host_name here in our struct (recursion safety)
return $obj_1x->{'__I2CONVERT_HOSTNAME'} if defined($obj_1x->{'__I2CONVERT_HOSTNAME'});
delete $obj_1x->{'__I2CONVERT_HOSTNAME'};
# if this object got what we want, return (it can be recursion and a template!)
if(defined($obj_1x->{'host_name'})) {
$obj_1x->{'__I2CONVERT_HOSTNAME'} = $obj_1x->{'host_name'};
return $obj_1x->{'__I2CONVERT_HOSTNAME'};
}
# we don't have a host name, should we look into a template?
# make sure _not_ to use
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
if (defined($obj_1x->{'__I2CONVERT_USES_TEMPLATE'}) && $obj_1x->{'__I2CONVERT_USES_TEMPLATE'} == 1) {
# get the object referenced as template - this is an array of templates, loop (funny recursion here)
foreach my $obj_1x_template (@{$obj_1x->{'__I2CONVERT_TEMPLATE_NAMES'}}) {
# get the template object associated with by its unique 'name' attr
my $obj_1x_tmpl = obj_get_tmpl_obj_by_tmpl_name($objs_1x, 'host', $obj_1x_template);
# now recurse into ourselves and look for a possible service_description
$host_name = obj_1x_get_host_host_name($objs_1x,$obj_1x_tmpl);
# bail here if search did not unveil anything
next if(!defined($host_name));
# get the host_name and return - first template wins
$obj_1x->{'__I2CONVERT_HOSTNAME'} = $host_name;
return $obj_1x->{'__I2CONVERT_HOSTNAME'};
}
}
# no template used, and no host name - broken object, ignore it
else {
return undef;
}
# we should never hit here
return undef;
}
# get host_name(s) from service object
sub obj_1x_get_service_host_name_arr {
# service objects may contain comma seperated host lists (ugly as ...)
my $objs_1x = shift;
my $obj_1x = shift;
my @host_name = ();
# if this object is invalid, bail early
return undef if !defined($obj_1x);
# first, check if we already got a host_name here in our struct (recursion safety)
return $obj_1x->{'__I2CONVERT_HOSTNAMES'} if defined($obj_1x->{'__I2CONVERT_HOSTNAMES'});
delete $obj_1x->{'__I2CONVERT_HOSTNAMES'};
# if this object got what we want, return (it can be recursion and a template!)
if(defined($obj_1x->{'host_name'})) {
#print "DEBUG: found $obj_1x->{'host_name'}\n";
# convert to array
delete($obj_1x->{'__I2CONVERT_HOSTNAMES'});
# check if host_name is a wildcard, or a possible comma seperated list
# using object tricks - http://docs.icinga.org/latest/en/objecttricks.html#objecttricks-service
if ($obj_1x->{'host_name'} =~ /^\*$/) {
@host_name = obj_1x_get_all_hostnames_arr($objs_1x);
} else {
@host_name = str2arr_by_delim_without_excludes($obj_1x->{'host_name'}, ',', 1);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
}
push @{$obj_1x->{'__I2CONVERT_HOSTNAMES'}}, @host_name;
#print "DEBUG: @{$obj_1x->{'__I2CONVERT_HOSTNAMES'}}";
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
return @host_name;
}
# we don't have a host name, should we look into a template?
# make sure _not_ to use
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
if (defined($obj_1x->{'__I2CONVERT_USES_TEMPLATE'}) && $obj_1x->{'__I2CONVERT_USES_TEMPLATE'} == 1) {
# get the object referenced as template - this is an array of templates, loop (funny recursion here)
foreach my $obj_1x_template (@{$obj_1x->{'__I2CONVERT_TEMPLATE_NAMES'}}) {
#say Dumper($obj_1x_template);
# get the template object associated with by its unique 'name' attr
my $obj_1x_tmpl = obj_get_tmpl_obj_by_tmpl_name($objs_1x, 'service', $obj_1x_template);
# now recurse into ourselves and look for all possible hostnames in array
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
@host_name = obj_1x_get_service_host_name_arr($objs_1x,$obj_1x_tmpl);
#print "DEBUG: from tmpl $obj_1x_template: " . join(" ", @host_name) . "\n";
# bail here if search did not unveil anything
next if(!@host_name);
# get the host_name and return - first template wins
# convert to array
delete($obj_1x->{'__I2CONVERT_HOSTNAMES'});
push @{$obj_1x->{'__I2CONVERT_HOSTNAMES'}}, @host_name;
return @host_name;
}
}
# no template used, and no host name - broken object, ignore it
else {
return undef;
}
# we should never hit here
return undef;
}
# get service_description from object
sub obj_1x_get_service_service_description {
my $objs_1x = shift;
my $obj_1x = shift;
my $host_name = shift;
my $service_description = "";
# if this object is invalid, bail early
return undef if !defined($obj_1x);
# first, check if we already got a service_description here in our struct (recursion safety)
return $obj_1x->{'__I2CONVERT_SERVICEDESCRIPTION'} if defined($obj_1x->{'__I2CONVERT_SERVICEDESCRIPTION'});
delete $obj_1x->{'__I2CONVERT_SERVICEDESCRIPTION'};
# if this object got what we want, return (it can be recursion and a template!)
if(defined($obj_1x->{'service_description'})) {
$obj_1x->{'__I2CONVERT_SERVICEDESCRIPTION'} = $obj_1x->{'service_description'};
return $obj_1x->{'__I2CONVERT_SERVICEDESCRIPTION'};
}
# we don't have a service description, should we look into a template?
# make sure _not_ to use
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
if (defined($obj_1x->{'__I2CONVERT_USES_TEMPLATE'}) && $obj_1x->{'__I2CONVERT_USES_TEMPLATE'} == 1) {
# get the object referenced as template - this is an array of templates, loop (funny recursion here)
foreach my $obj_1x_template (@{$obj_1x->{'__I2CONVERT_TEMPLATE_NAMES'}}) {
# get the template object associated with by its unique 'name' attr
my $obj_1x_tmpl = obj_get_tmpl_obj_by_tmpl_name($objs_1x, 'service', $obj_1x_template);
# now recurse into ourselves and look for a possible service_description
$service_description = obj_1x_get_service_service_description($objs_1x,$obj_1x_tmpl,$host_name); # we must pass the host_name
# bail here if search did not unveil anything
next if(!defined($service_description));
# get the service description and return - first template wins
$obj_1x->{'__I2CONVERT_SERVICEDESCRIPTION'} = $service_description;
return $obj_1x->{'__I2CONVERT_SERVICEDESCRIPTION'};
}
}
# no template used, and not service description - broken object, ignore it
else {
return undef;
}
# we should never hit here
return undef;
}
# get service_description from object
sub obj_1x_get_service_attr {
my $objs_1x = shift;
my $obj_1x = shift;
my $host_name = shift;
my $search_attr = shift;
my $service_attr = "";
# if this object is invalid, bail early
return undef if !defined($obj_1x);
# if this object got what we want, return (it can be recursion and a template!)
if(defined($obj_1x->{$search_attr})) {
return $obj_1x->{$search_attr};
}
# we don't have the attribute, should we look into a template?
# make sure _not_ to use
if (defined($obj_1x->{'__I2CONVERT_USES_TEMPLATE'}) && $obj_1x->{'__I2CONVERT_USES_TEMPLATE'} == 1) {
# get the object referenced as template - this is an array of templates, loop (funny recursion here)
foreach my $obj_1x_template (@{$obj_1x->{'__I2CONVERT_TEMPLATE_NAMES'}}) {
# get the template object associated with by its unique 'name' attr
my $obj_1x_tmpl = obj_get_tmpl_obj_by_tmpl_name($objs_1x, 'service', $obj_1x_template);
# now recurse into ourselves and look for a possible service_description
$service_attr = obj_1x_get_service_attr($objs_1x,$obj_1x_tmpl,$host_name,$search_attr); # we must pass the host_name and search_attr
#say Dumper($service_attr);
# bail here if search did not unveil anything
next if(!defined($service_attr));
# get the service attr and return - first template wins
return $service_attr;
}
}
# no template used, and not service description - broken object, ignore it
else {
return undef;
}
# we should never hit here
return undef;
}
# get service_description from object
sub obj_1x_get_host_attr {
my $objs_1x = shift;
my $obj_1x = shift;
my $host_name = shift;
my $search_attr = shift;
my $host_attr = "";
# if this object is invalid, bail early
return undef if !defined($obj_1x);
# if this object got what we want, return (it can be recursion and a template!)
if (defined($obj_1x->{$search_attr})) {
return $obj_1x->{$search_attr};
}
# we don't have the attribute, should we look into a template?
# make sure _not_ to use
if (defined($obj_1x->{'__I2CONVERT_USES_TEMPLATE'}) && $obj_1x->{'__I2CONVERT_USES_TEMPLATE'} == 1) {
# get the object referenced as template - this is an array of templates, loop (funny recursion here)
foreach my $obj_1x_template (@{$obj_1x->{'__I2CONVERT_TEMPLATE_NAMES'}}) {
# get the template object associated with by its unique 'name' attr
my $obj_1x_tmpl = obj_get_tmpl_obj_by_tmpl_name($objs_1x, 'host', $obj_1x_template);
# now recurse into ourselves and look for a possible service_description
$host_attr = obj_1x_get_host_attr($objs_1x,$obj_1x_tmpl,$host_name,$search_attr); # we must pass the host_name and search_attr
#say Dumper($service_attr);
# bail here if search did not unveil anything
next if(!defined($host_attr));
# get the service attr and return - first template wins
return $host_attr;
}
}
# no template used, and not service description - broken object, ignore it
else {
return undef;
}
# we should never hit here
return undef;
}
# get service_description from object
sub obj_1x_get_contact_attr {
my $objs_1x = shift;
my $obj_1x = shift;
my $search_attr = shift;
my $contact_attr = "";
# if this object is invalid, bail early
return undef if !defined($obj_1x);
# if this object got what we want, return (it can be recursion and a template!)
if(defined($obj_1x->{$search_attr})) {
return $obj_1x->{$search_attr};
}
# we don't have the attribute, should we look into a template?
# make sure _not_ to use
if (defined($obj_1x->{'__I2CONVERT_USES_TEMPLATE'}) && $obj_1x->{'__I2CONVERT_USES_TEMPLATE'} == 1) {
# get the object referenced as template - this is an array of templates, loop (funny recursion here)
foreach my $obj_1x_template (@{$obj_1x->{'__I2CONVERT_TEMPLATE_NAMES'}}) {
# get the template object associated with by its unique 'name' attr
my $obj_1x_tmpl = obj_get_tmpl_obj_by_tmpl_name($objs_1x, 'contact', $obj_1x_template);
# now recurse into ourselves and look for a possible contact attr
$contact_attr = obj_1x_get_contact_attr($objs_1x,$obj_1x_tmpl,$search_attr); # we must pass the search_attr
# bail here if search did not unveil anything
next if(!defined($contact_attr));
# get the contact attr and return - first template wins
return $contact_attr;
}
}
# no template used, and attr - broken object, ignore it
else {
return undef;
}
# we should never hit here
return undef;
}
# get servicegroups from object (already 2x and _array_ XXX)
sub obj_2x_get_service_servicegroups {
my $objs_2x = shift;
my $obj_2x = shift;
my $host_name = shift;
my $search_attr = shift;
my @service_groups;
#say Dumper("in obj_2x_get_service_attr");
# if this object is invalid, bail early
return undef if !defined($obj_2x);
# if this object got what we want, return (it can be recursion and a template!)
if(defined($obj_2x->{$search_attr}) && scalar(@{$obj_2x->{$search_attr}}) > 0) {
#say Dumper("in obj_2x_get_service_attr. found ");
#say Dumper($obj_2x->{$search_attr});
return @{$obj_2x->{$search_attr}};
}
# we don't have the attribute, should we look into a template?
# make sure _not_ to use
if (defined($obj_2x->{'__I2CONVERT_USES_TEMPLATE'}) && $obj_2x->{'__I2CONVERT_USES_TEMPLATE'} == 1) {
# get the object referenced as template - this is an array of templates, loop (funny recursion here)
foreach my $obj_2x_template (@{$obj_2x->{'__I2CONVERT_TEMPLATE_NAMES'}}) {
#say Dumper("in obj_2x_get_service_attr template");
#say Dumper($obj_2x_template);
# get the template object associated with by its unique 'name' attr
my $obj_2x_tmpl = obj_get_tmpl_obj_by_tmpl_name($objs_2x, 'service', $obj_2x_template);
#say Dumper($obj_2x_tmpl);
# now recurse into ourselves and look for a possible service_description
push @service_groups, obj_2x_get_service_servicegroups($objs_2x,$obj_2x_tmpl,$host_name,$search_attr); # we must pass the host_name and search_attr
#say Dumper($service_attr);
# bail here if search did not unveil anything
next if(scalar(@service_groups) == 0);
# get the service attr and return - first template wins
return @service_groups;
}
}
# no template used, and not service description - broken object, ignore it
else {
return undef;
}
# we should never hit here
return undef;
}
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
################################################################################
# Conversion
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
#################################################################################
# convert CVs, *_url, etc into custom hash
sub obj_convert_custom_attr_to_custom_hash {
my $obj = shift;
my $custom;
foreach my $key (keys %{$obj}) {
if ($key =~ /^_|_url$|notes|image|coords/ && $key !~ /__I2CONVERT/) {
my $new_key = ($key =~ /^\d/) ? "\"$key\"" : $key;
$custom->{$new_key} = escape_str($obj->{$key});
}
}
$obj->{'__I2CONVERT_CUSTOM_ATTR'} = $custom;
return $custom;
}
sub resolve_macro_attribute {
my $cfg_obj = shift;
my $obj = shift;
my $macro_name = shift;
my $macro_value;
my $attr_name;
my $obj_host;
my $host_name;
# NOTE: BIG FAT NOTE: we only support static configuration macros as arguments
# we cannot resolve macros which require the full configuration in memory, but
# only local service (or host related macros) for command arguments
# _ == custom var from host/service
if ($macro_name =~ /^_HOST(\w+)/) {
$attr_name = "_$1";
#debug("MACRO RESOLVER: found Host CV '$macro_name'.");
if (defined($obj->{'__I2CONVERT_HOSTNAME'})) {
$host_name = $obj->{'__I2CONVERT_HOSTNAME'};
$obj_host = $obj;
} else {
$host_name = $obj->{'__I2CONVERT_SERVICE_HOSTNAME'};
# first grab the according host object (this is a service object)
$obj_host = obj_get_host_obj_by_host_name($cfg_obj, $host_name);
}
$macro_value = obj_1x_get_host_attr($cfg_obj, $obj_host, $host_name, $attr_name);
#debug("MACRO RESOLVER: found $attr_name with value '$macro_value' on " . Dumper($obj_host));
return $macro_value;
}
elsif ($macro_name =~ /^_SERVICE(\w+)/) {
$attr_name = "_$1";
# if this is a host object, this macro is invalid! XXX
return undef if defined($obj->{'__I2CONVERT_HOSTNAME'});
#debug("MACRO RESOLVER: found Service CV '$macro_name'.");
$macro_value = obj_1x_get_service_attr($cfg_obj, $obj, $obj->{'__I2CONVERT_SERVICE_HOSTNAME'}, $attr_name);
#debug("MACRO RESOLVER: found $attr_name with value '$macro_value' on " . Dumper($obj));
return $macro_value;
}
else {
# XXX this is way too modular to actually work with macros not having any underscores and other magic
if ($macro_name =~ /^HOST(\w+)/) {
$attr_name = lc $1;
#debug("MACRO RESOLVER: found Host attribute '$macro_name'.");
if (defined($obj->{'__I2CONVERT_HOSTNAME'})) {
$host_name = $obj->{'__I2CONVERT_HOSTNAME'};
$obj_host = $obj;
} else {
$host_name = $obj->{'__I2CONVERT_SERVICE_HOSTNAME'};
# first grab the according host object (this is a service object)
$obj_host = obj_get_host_obj_by_host_name($cfg_obj, $host_name);
}
$macro_value = obj_1x_get_host_attr($cfg_obj, $obj_host, $host_name, $attr_name);
#debug("MACRO RESOLVER: found $attr_name with value '$macro_value' on " . Dumper($obj_host));
return $macro_value;
}
elsif ($macro_name =~ /^SERVICE(\w+)/) {
$attr_name = lc $1;
# if this is a host object, this macro is invalid! XXX
return undef if defined($obj->{'__I2CONVERT_HOSTNAME'});
#debug("MACRO RESOLVER: found Service attribute '$macro_name'.");
$macro_value = obj_1x_get_service_attr($cfg_obj, $obj, $obj->{'__I2CONVERT_SERVICE_HOSTNAME'}, $attr_name);
#debug("MACRO RESOLVER: found $attr_name with value '$macro_value' on " . Dumper($obj));
return $macro_value;
}
elsif ($macro_name =~ /^CONTACT(\w+)/) {
$attr_name = lc $1;
# XXX is that possible from command arguments?
# http://docs.icinga.org/latest/en/macrolist.html#availabilitychart (NO)
}
}
return undef;
}
sub resolve_macros {
my $cfg_obj = shift;
my $obj = shift;
my $str = shift;
if ($str =~ /^\$(\w+)\$$/) {
return resolve_macro_attribute($cfg_obj, $obj, $1);
} else {
return $str;
}
}
# convert notification_options to state|type_filter
sub convert_notification_options_to_filter {
my $notification_options = shift;
my $filter = ();
@{$filter->{'state'}} = ();
@{$filter->{'type'}} = ();
# define all types
my $filter_names = {
'o' => 'StateFilterOK',
'w' => 'StateFilterWarning',
'c' => 'StateFilterCritical',
'u' => 'StateFilterUnknown',
'd' => 'StateFilterCritical', # down, treated as critical
's' => 'NotificationFilterDowntimeStart | NotificationFilterDowntimeEnd | NotificationFilterDowntimeRemoved',
'r' => 'NotificationFilterRecovery',
'f' => 'NotificationFilterFlappingStart | NotificationFilterFlappingEnd'
};
my $filter_by = {
'o' => 'state',
'w' => 'state',
'c' => 'state',
'u' => 'state',
'd' => 'state',
's' => 'type',
'r' => 'type',
'f' => 'type'
};
# split the string
my @options = str2arr_by_delim_without_excludes($notification_options, ',', 1);
# verify if there's 'n' (none) and ignore the rest then
if (grep /n/, @options) {
push @{$filter->{'state'}}, 0;
push @{$filter->{'type'}}, 0;
return $filter;
}
# recovery requires state up
if (grep /r/, @options) {
push @{$filter->{'state'}}, 'StateFilterOK';
}
# always add NotificationFilterProblem|Custom
push @{$filter->{'type'}}, 'NotificationFilterProblem';
push @{$filter->{'type'}}, 'NotificationFilterCustom';
# verify if there's 'a' (all) and add all filters
if (grep /a/, @options) {
foreach my $by (keys %{$filter_by}) {
push @{$filter->{$filter_by->{$by}}}, $filter_names->{$by};
}
return $filter;
}
# the selective way
foreach my $option (@options) {
push @{$filter->{$filter_by->{$option}}}, $filter_names->{$option};
}
return $filter;
}
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# host|service_notification_commands are a comma seperated list w/o arguments
sub convert_notificationcommand {
my $objs_1x = shift;
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
my $commands_1x = shift;
my $obj_1x = shift;
my $global_macros_1x = shift;
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
my $command_name_1x;
my @commands = ();
my $notification_commands_2x = ();
my $host_notification_commands;
my $service_notification_commands;
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# bail early if this is not a valid contact object
return undef if (!defined($obj_1x->{'contact_name'}));
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# bail early if required commands not available (not a valid 1.x object either)
if (defined($obj_1x->{'host_notification_commands'})) {
$host_notification_commands = $obj_1x->{'host_notification_commands'};
}
else {
# look in the template
$host_notification_commands = obj_1x_get_contact_attr($objs_1x,$obj_1x,'host_notification_commands');
}
if (defined($obj_1x->{'service_notification_commands'})) {
$service_notification_commands = $obj_1x->{'service_notification_commands'};
}
else {
# look in the template
$service_notification_commands = obj_1x_get_contact_attr($objs_1x,$obj_1x,'service_notification_commands');
}
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# a contact has a comma seperated list of notification commands by host and service
my $all_notification_commands = {};
push @{$all_notification_commands->{'host'}}, split /,\s+/, $host_notification_commands;
push @{$all_notification_commands->{'service'}}, split /,\s+/, $service_notification_commands;
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
foreach my $obj_notification_command_key ( keys %{$all_notification_commands}) {
# fetch all command names in array via type key
my @notification_commands = @{$all_notification_commands->{$obj_notification_command_key}};
my $notification_command_type = $obj_notification_command_key;
foreach my $notification_command (@notification_commands) {
# now back in all command objects of 1.x
foreach my $command_1x_key (keys %{$commands_1x}) {
chomp $notification_command; # remove trailing spaces
if ($commands_1x->{$command_1x_key}->{'command_name'} eq $notification_command) {
# save the type (host, service) and then by command name
$notification_commands_2x->{$notification_command_type}->{$notification_command} = escape_str($commands_1x->{$command_1x_key}->{'command_line'});
#say Dumper($commands_1x->{$command_1x_key});
# flag this 1.x command as being used
$commands_1x->{$command_1x_key}->{__I2_CONVERT_NOTIFICATION_COMMAND_USED} = 1;
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
}
}
}
}
return $notification_commands_2x;
}
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# event_handler
sub convert_eventhandler {
my $commands_1x = shift;
my $obj_1x = shift;
my $global_macros_1x = shift;
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
my $command_name_1x;
my @commands = ();
my $event_commands_2x = ();
# bail early if required commands not available (not a valid 1.x object either)
return if (!defined($obj_1x->{'event_handler'}));
my $event_command = $obj_1x->{'event_handler'};
#say Dumper($event_command);
# now back in all command objects of 1.x
foreach my $command_1x_key (keys %{$commands_1x}) {
chomp $event_command; # remove trailing spaces
if ($commands_1x->{$command_1x_key}->{'command_name'} eq $event_command) {
# save the command line and command name
$event_commands_2x->{'command_name'} = $event_command;
$event_commands_2x->{'command_line'} = escape_str($commands_1x->{$command_1x_key}->{'command_line'});
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
}
}
return $event_commands_2x;
}
# check_command accepts argument parameters, special treatment
sub convert_checkcommand {
my $cfg_obj_1x = shift;
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
my $commands_1x = shift;
my $obj_1x = shift; #host or service
my $service_hostname = shift;
my $global_macros_1x = shift;
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
my $command_1x;
my $command_2x = {};
my $check_command;
#say Dumper($commands_1x);
#say Dumper($obj_1x);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# objects without check_command may have it defined in template!
if (!defined($obj_1x->{'check_command'})) {
# service
if (defined($obj_1x->{'__I2CONVERT_SERVICEDESCRIPTION'})) {
$check_command = obj_1x_get_service_attr($cfg_obj_1x, $obj_1x, $service_hostname, 'check_command');
} else {
$check_command = obj_1x_get_host_attr($cfg_obj_1x, $obj_1x, $service_hostname, 'check_command');
}
} else {
$check_command = $obj_1x->{'check_command'};
}
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
#debug("check_command: $obj_1x->{'check_command'}" );
# split by ! and take only the check command
my ($real_command_name_1x, @command_args_1x) = split /!/, $check_command;
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# ignore objects with empty check_command attribute
#return if (!defined($real_command_name_1x));
#debug("1x Command Name: $real_command_name_1x");
if (@command_args_1x) {
#debug("1x Command Args: @command_args_1x");
}
foreach my $command_1x_key (keys %{$commands_1x}) {
#say Dumper($commands_1x->{$command_1x_key}->{'command_name'});
if ($commands_1x->{$command_1x_key}->{'command_name'} eq $real_command_name_1x) {
#debug("Found: $real_command_name_1x");
# save the command_line
$command_2x->{'check_command'} = escape_str($commands_1x->{$command_1x_key}->{'command_line'});
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
$command_2x->{'check_command_name_1x'} = $real_command_name_1x;
#debug("2x Command: $command_2x->{'check_command'}");
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
}
}
return ($command_2x, @command_args_1x);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
}
# convert existing 1x objects into the 2x objects hive
sub migrate_2x {
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# v1 -> v2
# register 0 == template
# use == inherits template
# dependency == ...
# escalation == ...
# hashref is selectable by type first
my $icinga2_cfg = shift;
my $cfg_obj_1x = shift;
my $cfg_obj_cache_1x = shift;
my $global_macros_1x = shift;
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# build a new hashref with the actual 2.x config inside
my $cfg_obj_2x = {};
my $command_obj_cnt = 0;
# stats
my $count = {};
my $count_original_services;
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_NAME'} = '__I2CONVERT_COMMAND_DUMMY';
######################################
# SERVICE
# do the magic lookup for host_name/
# service_description for each object
# only once
######################################
my $service_cnt = 0;
foreach my $service_obj_1x_key (keys %{@$cfg_obj_1x{'service'}}) {
#say Dumper(@$cfg_obj_1x{'service'}->{$service_obj_1x_key});
my $obj_1x_service = @$cfg_obj_1x{'service'}->{$service_obj_1x_key};
####################################################
# verify template is/use
####################################################
$obj_1x_service->{'__I2CONVERT_IS_TEMPLATE'} = obj_1x_is_template($obj_1x_service);
$obj_1x_service->{'__I2CONVERT_USES_TEMPLATE'} = obj_1x_uses_template($obj_1x_service);
$obj_1x_service->{'__I2CONVERT_TEMPLATE_NAME'} = $obj_1x_service->{'name'};
# this can be a comma seperated list of templates
my @service_templates = ();
if(defined($obj_1x_service->{'use'})) {
@service_templates = str2arr_by_delim_without_excludes($obj_1x_service->{'use'}, ',', 1);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
}
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
push @{$obj_1x_service->{'__I2CONVERT_TEMPLATE_NAMES'}}, @service_templates;
# add dependency to ITL template to objects
if ($obj_1x_service->{'__I2CONVERT_IS_TEMPLATE'} == 0) {
if(defined($icinga2_cfg->{'itl'}->{'service-template'}) && $icinga2_cfg->{'itl'}->{'service-template'} ne "") {
push @{$obj_1x_service->{'__I2CONVERT_TEMPLATE_NAMES'}}, $icinga2_cfg->{'itl'}->{'service-template'};
$obj_1x_service->{'__I2CONVERT_USES_TEMPLATE'} = 1;
}
}
####################################################
# migrate custom attributes
####################################################
my $custom = obj_convert_custom_attr_to_custom_hash($obj_1x_service);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
####################################################
# get related host_name/service_description
# used later in host->service resolval
####################################################
# XXX even if the service object uses templates, we need to figure out its host_name/service_description in order to safely link hosts towards it
my @host_names = obj_1x_get_service_host_name_arr($cfg_obj_1x, $obj_1x_service);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
#print "DEBUG: service @host_names\n";
delete($obj_1x_service->{'__I2CONVERT_HOSTNAMES'});
if(@host_names == 0) {
# set a dummy value for postprocessing - we need the prepared 2.x service for later object tricks
push @host_names, "__I2CONVERT_DUMMY";
}
push @{$obj_1x_service->{'__I2CONVERT_HOSTNAMES'}}, @host_names;
# if there is more than one host_name involved on the service object, clone it in a loop
foreach my $service_host_name (@{$obj_1x_service->{'__I2CONVERT_HOSTNAMES'}}) {
# we can only look up services with their uniqueness to the host_name
$obj_1x_service->{'__I2CONVERT_SERVICEDESCRIPTION'} = obj_1x_get_service_service_description($cfg_obj_1x, $obj_1x_service, $service_host_name);
#say Dumper($obj_1x_service);
# skip non-template objects without a valid service description (we cannot tolerate 'name' here!)
# XXX find a better way - we actually need all services in the list, even if __I2CONVERT_SERVICEDESCRIPTION is undef
if (!defined($obj_1x_service->{'__I2CONVERT_SERVICEDESCRIPTION'}) && $obj_1x_service->{'__I2CONVERT_IS_TEMPLATE'} == 0) {
#debug("Skipping invalid service object without service_description ".Dumper($obj_1x_service));
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
next;
}
####################################################
# clone service object into 2.x
####################################################
$cfg_obj_2x->{'service'}->{$service_cnt} = dclone(@$cfg_obj_1x{'service'}->{$service_obj_1x_key});
# immediately overwrite the correct host_name - if there's no dummy value set for further processing
if ($service_host_name !~ /__I2CONVERT_DUMMY/) {
$cfg_obj_2x->{'service'}->{$service_cnt}->{__I2CONVERT_SERVICE_HOSTNAME} = $service_host_name;
}
#say Dumper($cfg_obj_2x->{'service'}->{$service_cnt});
####################################################
# map existing service attributes
# same:
# - display_name
# - max_check_attempts
# - check_period
# - notification_period
# custom:
# - _CV
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# - action_url
# - notes_url
# - notes
# - icon_image
# change:
# - servicegroups (commaseperated strings to array)
# - check_command
# - check_interval (X min -> Xm) + normal_check_interval
# - retry_interval (X min -> Xm) + retry_check_interval
# - notification_interval (X min -> Xm)
# - contacts => users XXX DO NOT DELETE contacts and contactgroups, they will be assembled later for notifications!
####################################################
####################################################
# migrate custom attributes
####################################################
$cfg_obj_2x->{'service'}->{$service_cnt}->{'__I2CONVERT_CUSTOM_ATTR'} = obj_convert_custom_attr_to_custom_hash($cfg_obj_2x->{'service'}->{$service_cnt});
#say Dumper($cfg_obj_2x->{'service'}->{$service_cnt}->{'__I2CONVERT_CUSTOM_ATTR'});
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
##########################################
# servicegroups
##########################################
delete($cfg_obj_2x->{'service'}->{$service_cnt}->{'servicegroups'});
# debug #
@{$cfg_obj_2x->{'service'}->{$service_cnt}->{'servicegroups'}} = ();
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
if(defined($obj_1x_service->{'servicegroups'})) {
# check if there's additive inheritance required, and save a flag
if ($obj_1x_service->{'servicegroups'} =~ /^\+/) {
$cfg_obj_2x->{'service'}->{$service_cnt}->{'__I2_CONVERT_SG_ADD'} = 1;
$obj_1x_service->{'servicegroups'} =~ s/^\+//;
}
# convert comma seperated list to array
push @{$cfg_obj_2x->{'service'}->{$service_cnt}->{'servicegroups'}}, str2arr_by_delim_without_excludes($obj_1x_service->{'servicegroups'}, ',', 1);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
#print "DEBUG: servicegroups " . join (" ", @{$cfg_obj_2x->{'service'}->{$service_cnt}->{'servicegroups'}});
}
#say Dumper($cfg_obj_2x->{'service'}->{$service_cnt}->{__I2CONVERT_SERVICE_HOSTNAME});
#say Dumper($cfg_obj_2x->{'service'}->{$service_cnt}->{'servicegroups'});
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
##########################################
# check_interval
##########################################
my $service_check_interval = undef;
if(defined($obj_1x_service->{'normal_check_interval'})) {
$service_check_interval = $obj_1x_service->{'normal_check_interval'};
}
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
if(defined($obj_1x_service->{'check_interval'})) {
$service_check_interval = $obj_1x_service->{'check_interval'};
}
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# we assume that 1.x kept 1m default interval, and map it
if (defined($service_check_interval)) {
$cfg_obj_2x->{'service'}->{$service_cnt}->{'check_interval'} = $service_check_interval."m";
}
##########################################
# retry_interval
##########################################
my $service_retry_interval = undef;
if(defined($obj_1x_service->{'retry_check_interval'})) {
$service_retry_interval = $obj_1x_service->{'retry_check_interval'};
}
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
if(defined($obj_1x_service->{'retry_interval'})) {
$service_retry_interval = $obj_1x_service->{'retry_interval'};
}
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# we assume that 1.x kept 1m default interval, and map it
if (defined($service_retry_interval)) {
$cfg_obj_2x->{'service'}->{$service_cnt}->{'retry_interval'} = $service_retry_interval."m";
}
##########################################
# notification_*
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
##########################################
my $service_notification_interval = undef;
if(defined($obj_1x_service->{'notification_interval'})) {
$service_notification_interval = $obj_1x_service->{'notification_interval'};
} else {
$service_notification_interval = obj_1x_get_service_attr($cfg_obj_1x, $obj_1x_service, $service_host_name, 'notification_interval');
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
}
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# we assume that 1.x kept 1m default interval, and map it
if (defined($service_notification_interval)) {
$cfg_obj_2x->{'service'}->{$service_cnt}->{'notification_interval'} = $service_notification_interval."m";
}
if(defined($obj_1x_service->{'notification_options'})) {
$cfg_obj_2x->{'service'}->{$service_cnt}->{'__I2CONVERT_NOTIFICATION_FILTERS'} = convert_notification_options_to_filter($obj_1x_service->{'notification_options'});
#say Dumper($cfg_obj_2x->{'service'}->{$service_cnt});
} else {
# fetch them from template tree for later notification object mapping
my $notification_options = obj_1x_get_service_attr($cfg_obj_1x, $obj_1x_service, $service_host_name, 'notification_options');
$cfg_obj_2x->{'service'}->{$service_cnt}->{'__I2CONVERT_NOTIFICATION_FILTERS'} = convert_notification_options_to_filter($notification_options);
}
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
##########################################
# eventhandler
##########################################
if (defined($obj_1x_service->{'event_handler'})) {
my $service_event_command_2x = convert_eventhandler(@$cfg_obj_1x{'command'}, $obj_1x_service, $global_macros_1x);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
#say Dumper($service_event_command_2x);
# XXX do not add duplicate event commands, they must remain unique by their check_command origin!
my $command_type = 'Event';
if (obj_2x_command_exists($cfg_obj_2x, $obj_1x_service->{'event_handler'}, $command_type) != 1) {
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# create a new EventCommand 2x object with the original name
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_TYPE'} = 'Event';
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_NAME'} = $service_event_command_2x->{'command_name'};
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_LINE'} = $service_event_command_2x->{'command_line'};
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_MACROS'} = $service_event_command_2x->{'command_macros'};
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# use the ITL plugin check command template
if(defined($icinga2_cfg->{'itl'}->{'eventcommand-template'}) && $icinga2_cfg->{'itl'}->{'eventcommand-template'} ne "") {
push @{$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_TEMPLATE_NAMES'}}, $icinga2_cfg->{'itl'}->{'eventcommand-template'};
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_USES_TEMPLATE'} = 1;
}
# our PK
$command_obj_cnt++;
}
# the event_handler name of 1.x is still the unique command object name, so we just keep
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# in __I2_CONVERT_EVENTCOMMAND_NAME in our service object
$cfg_obj_2x->{'service'}->{$service_cnt}->{'__I2_CONVERT_EVENTCOMMAND_NAME'} = $service_event_command_2x->{'command_name'};
}
##########################################
# volatile is bool only
##########################################
if (defined($obj_1x_service->{'is_volatile'})) {
$cfg_obj_2x->{'service'}->{$service_cnt}->{'volatile'} = ($obj_1x_service->{'is_volatile'} > 0) ? 1 : 0;
}
##########################################
# map the service check_command to 2.x
##########################################
my ($service_check_command_2x, @command_args_1x) = convert_checkcommand($cfg_obj_1x, @$cfg_obj_1x{'command'}, $obj_1x_service, $service_host_name, $global_macros_1x);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
#say Dumper($service_check_command_2x);
if (defined($service_check_command_2x->{'check_command_name_1x'})) {
# XXX do not add duplicate check commands, they must remain unique by their check_command origin!
my $command_type = 'Check';
if (obj_2x_command_exists($cfg_obj_2x, $service_check_command_2x->{'check_command_name_1x'}, $command_type) != 1) {
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# create a new CheckCommand 2x object with the original name
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_TYPE'} = 'Check';
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_NAME'} = $service_check_command_2x->{'check_command_name_1x'};
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_LINE'} = $service_check_command_2x->{'check_command'};
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_MACROS'} = $service_check_command_2x->{'command_macros'};
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# use the ITL plugin check command template
if(defined($icinga2_cfg->{'itl'}->{'checkcommand-template'}) && $icinga2_cfg->{'itl'}->{'checkcommand-template'} ne "") {
push @{$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_TEMPLATE_NAMES'}}, $icinga2_cfg->{'itl'}->{'checkcommand-template'};
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_USES_TEMPLATE'} = 1;
}
# add the command macros to the service 2x object
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
if(defined($service_check_command_2x->{'command_macros'})) {
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_MACROS'} = dclone($service_check_command_2x->{'command_macros'});
}
# our PK
$command_obj_cnt++;
}
# make sure service object still got the checkcommand assigned
# the check command name of 1.x is still the unique command object name, so we just keep
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# in $service_check_command_2x->{'check_command'} the cut real check_command_name_1x
delete($service_check_command_2x->{'check_command'});
$cfg_obj_2x->{'service'}->{$service_cnt}->{'__I2_CONVERT_CHECKCOMMAND_NAME'} = $service_check_command_2x->{'check_command_name_1x'};
}
# save all command args as macros
my $arg_cnt = 1;
foreach my $command_arg_1x (@command_args_1x) {
$obj_1x_service->{__I2CONVERT_SERVICE_HOSTNAME} = $service_host_name;
my $command_arg_2x = resolve_macros($cfg_obj_1x, $obj_1x_service, $command_arg_1x);
$obj_1x_service->{__I2CONVERT_SERVICE_HOSTNAME} = undef;
@$cfg_obj_2x{'service'}->{$service_cnt}->{'__I2CONVERT_MACROS'}->{"ARG" . $arg_cnt} = escape_str($command_arg_2x);
$arg_cnt++;
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
}
# our PK
$service_cnt++;
}
}
######################################
# HOST
# use => inherit template
# register 0 => template
# check_command => create a new service?
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
######################################
# "get all 'host' hashref as array in hashmap, and their keys to access it"
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
foreach my $host_obj_1x_key (keys %{@$cfg_obj_1x{'host'}}) {
#say Dumper(@$cfg_obj_1x{'host'}->{$host_obj_1x_key});
my $obj_1x_host = @$cfg_obj_1x{'host'}->{$host_obj_1x_key};
####################################################
# verify template is/use
####################################################
$obj_1x_host->{'__I2CONVERT_IS_TEMPLATE'} = obj_1x_is_template($obj_1x_host);
$obj_1x_host->{'__I2CONVERT_USES_TEMPLATE'} = obj_1x_uses_template($obj_1x_host);
$obj_1x_host->{'__I2CONVERT_TEMPLATE_NAME'} = $obj_1x_host->{'name'};
# this can be a comma seperated list of templates
my @host_templates = ();
if(defined($obj_1x_host->{'use'})) {
@host_templates = str2arr_by_delim_without_excludes($obj_1x_host->{'use'}, ',', 1);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
}
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
push @{$obj_1x_host->{'__I2CONVERT_TEMPLATE_NAMES'}}, @host_templates;
####################################################
# get the related host_name
####################################################
# XXX even if the host object uses templates, we need to figure out its host_name in order to safely link services towards it
$obj_1x_host->{'__I2CONVERT_HOSTNAME'} = obj_1x_get_host_host_name($cfg_obj_1x, $obj_1x_host);
####################################################
# skip objects without a valid hostname
####################################################
if (!defined($obj_1x_host->{'__I2CONVERT_HOSTNAME'}) && $obj_1x_host->{'__I2CONVERT_IS_TEMPLATE'} == 0) {
#debug("Skipping invalid host object without host_name ".Dumper($obj_1x_host));
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
next;
}
#say Dumper($obj_1x_host);
#
# FIXME do that later on
# primary host->service relation resolval
####################################################
# Clone the existing object into 2.x
####################################################
# save a copy with the valid ones
$cfg_obj_2x->{'host'}->{$host_obj_1x_key} = dclone(@$cfg_obj_1x{'host'}->{$host_obj_1x_key});
####################################################
# map existing host attributes
# same:
# - max_check_attempts
# - check_period
# - notification_period
# custom:
# - _CVs
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# - action_url
# - notes_url
# - notes
# - icon_image
# - statusmap_image
# change:
# - display_name (if alias is set, overwrites it)
# - hostgroups (commaseperated strings to array)
# - check_interval (X min -> Xm) + normal_check_interval
# - retry_interval (X min -> Xm) + retry_check_interval
# - notification_interval (X min -> Xm)
# - contacts => users XXX DO NOT DELETE contacts and contactgroups - they will be assembled later for notifications!
# -
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
####################################################
##########################################
# macros (address*, etc)
##########################################
if(defined($cfg_obj_2x->{'host'}->{$host_obj_1x_key}->{'address'})) {
$cfg_obj_2x->{'host'}->{$host_obj_1x_key}->{'__I2CONVERT_MACROS'}->{'address'} = $cfg_obj_2x->{'host'}->{$host_obj_1x_key}->{'address'};
}
if(defined($cfg_obj_2x->{'host'}->{$host_obj_1x_key}->{'address6'})) {
$cfg_obj_2x->{'host'}->{$host_obj_1x_key}->{'__I2CONVERT_MACROS'}->{'address6'} = $cfg_obj_2x->{'host'}->{$host_obj_1x_key}->{'address6'};
}
####################################################
# migrate custom attributes
####################################################
$cfg_obj_2x->{'host'}->{$host_obj_1x_key}->{'__I2CONVERT_CUSTOM_ATTR'} = obj_convert_custom_attr_to_custom_hash($cfg_obj_2x->{'host'}->{$host_obj_1x_key});
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
####################################################
# display_name -> alias mapping
####################################################
# if there was an host alias defined, make this the primary display_name for 2x
if(defined($obj_1x_host->{'alias'})) {
$cfg_obj_2x->{'host'}->{$host_obj_1x_key}->{'display_name'} = escape_str($obj_1x_host->{'alias'});
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
}
##########################################
# hostgroups
##########################################
delete($cfg_obj_2x->{'host'}->{$host_obj_1x_key}->{'hostgroups'});
if(defined($obj_1x_host->{'hostgroups'})) {
# check if there's additive inheritance required, and save a flag
if ($obj_1x_host->{'hostgroups'} =~ /^\+/) {
$cfg_obj_2x->{'host'}->{$host_obj_1x_key}->{'__I2_CONVERT_HG_ADD'} = 1;
$obj_1x_host->{'hostgroups'} =~ s/^\+//;
}
# convert comma seperated list to array
push @{$cfg_obj_2x->{'host'}->{$host_obj_1x_key}->{'hostgroups'}}, str2arr_by_delim_without_excludes($obj_1x_host->{'hostgroups'}, ',', 1);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
#print "DEBUG: hostgroups " . join (" ", @{$cfg_obj_2x->{'host'}->{$host_obj_1x_key}->{'hostgroups'}});
}
##########################################
# check_interval
##########################################
my $host_check_interval = undef;
if(defined($obj_1x_host->{'normal_check_interval'})) {
$host_check_interval = $obj_1x_host->{'normal_check_interval'};
}
if(defined($obj_1x_host->{'check_interval'})) {
$host_check_interval = $obj_1x_host->{'check_interval'};
}
# we assume that 1.x kept 1m default interval, and map it
if (defined($host_check_interval)) {
$cfg_obj_2x->{'host'}->{$host_obj_1x_key}->{'check_interval'} = $host_check_interval."m";
}
##########################################
# retry_interval
##########################################
my $host_retry_interval = undef;
if(defined($obj_1x_host->{'retry_check_interval'})) {
$host_retry_interval = $obj_1x_host->{'retry_check_interval'};
}
if(defined($obj_1x_host->{'retry_interval'})) {
$host_retry_interval = $obj_1x_host->{'retry_interval'};
}
# we assume that 1.x kept 1m default interval, and map it
if (defined($host_retry_interval)) {
$cfg_obj_2x->{'host'}->{$host_obj_1x_key}->{'retry_interval'} = $host_retry_interval."m";
}
##########################################
# notification_interval
##########################################
my $host_notification_interval = undef;
if(defined($obj_1x_host->{'notification_interval'})) {
$host_notification_interval = $obj_1x_host->{'notification_interval'};
} else {
$host_notification_interval = obj_1x_get_host_attr($cfg_obj_1x, $obj_1x_host, 'notification_interval');
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
}
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# we assume that 1.x kept 1m default interval, and map it
if (defined($host_notification_interval)) {
$cfg_obj_2x->{'host'}->{$host_obj_1x_key}->{'notification_interval'} = $host_notification_interval."m";
}
if(defined($obj_1x_host->{'notification_options'})) {
$cfg_obj_2x->{'host'}->{$host_obj_1x_key}->{'__I2CONVERT_NOTIFICATION_FILTERS'} = convert_notification_options_to_filter($obj_1x_host->{'notification_options'});
#say Dumper($cfg_obj_2x->{'host'}->{$host_obj_1x_key});
} else {
# fetch them from template tree for later notification object mapping
my $notification_options = obj_1x_get_host_attr($cfg_obj_1x, $obj_1x_host, 'notification_options');
$cfg_obj_2x->{'host'}->{$host_obj_1x_key}->{'__I2CONVERT_NOTIFICATION_FILTERS'} = convert_notification_options_to_filter($notification_options);
}
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
if(defined($obj_1x_host->{'parents'})) {
my @host_parents = str2arr_by_delim_without_excludes($obj_1x_host->{'parents'}, ',', 1);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
push @{$cfg_obj_2x->{'host'}->{$host_obj_1x_key}->{'__I2CONVERT_PARENT_HOSTNAMES'}}, @host_parents;
}
####################################################
# Icinga 2 Hosts don't have a check_command anymore
# - get a similar service with command_name lookup
# and link that service
####################################################
my ($host_check_command_2x, @command_args_1x) = convert_checkcommand($cfg_obj_1x, @$cfg_obj_1x{'command'}, $obj_1x_host, $obj_1x_host->{'__I2CONVERT_HOSTNAME'}, $global_macros_1x);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
#say Dumper($host_check_command_2x);
if(defined($host_check_command_2x->{'check_command_name_1x'})) {
# XXX TODO match on the command_name in available services for _this_ host later on. right on, just save it
$cfg_obj_2x->{'host'}->{$host_obj_1x_key}->{'__I2CONVERT_HOSTCHECK_NAME'} = $host_check_command_2x->{'check_command_name_1x'};
}
# XXX skip all host templates, they do not need to be linked with services!
#if ($cfg_obj_2x->{'host'}->{$host_obj_1x_key}->{'__I2CONVERT_IS_TEMPLATE'} == 1) {
# debug("Skipping host template for linking against service.");
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# next;
#}
# NOTE: the relation between host and services for 2x will be done later
# this is due to the reason we may manipulate service objects later
# e.g. when relinking the servicegroup members, etc
# otherwise we would have to make sure to update 2 locations everytime
}
######################################
# CONTACT => USER
######################################
my $user_cnt = 0;
if (!@$cfg_obj_1x{'contact'}) {
goto SKIP_CONTACTS;
}
foreach my $contact_obj_1x_key (keys %{@$cfg_obj_1x{'contact'}}) {
my $obj_1x_contact = @$cfg_obj_1x{'contact'}->{$contact_obj_1x_key};
####################################################
# verify template is/use
####################################################
$obj_1x_contact->{'__I2CONVERT_IS_TEMPLATE'} = obj_1x_is_template($obj_1x_contact);
$obj_1x_contact->{'__I2CONVERT_USES_TEMPLATE'} = obj_1x_uses_template($obj_1x_contact);
$obj_1x_contact->{'__I2CONVERT_TEMPLATE_NAME'} = $obj_1x_contact->{'name'}; # XXX makes sense when IS_TEMPLATE is set
# this can be a comma seperated list of templates
my @contact_templates = ();
if(defined($obj_1x_contact->{'use'})) {
@contact_templates = str2arr_by_delim_without_excludes($obj_1x_contact->{'use'}, ',', 1);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
}
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
push @{$obj_1x_contact->{'__I2CONVERT_TEMPLATE_NAMES'}}, @contact_templates;
####################################################
# get all notification commands
####################################################
my $notification_commands_2x = convert_notificationcommand($cfg_obj_1x, @$cfg_obj_1x{'command'}, $obj_1x_contact, $global_macros_1x);
#say Dumper($obj_1x_contact);
#say Dumper($notification_commands_2x);
#say Dumper("======================================");
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# clone it into our users hash
$cfg_obj_2x->{'user'}->{$contact_obj_1x_key} = dclone(@$cfg_obj_1x{'contact'}->{$contact_obj_1x_key});
# set our own __I2CONVERT_TYPE
$cfg_obj_2x->{'user'}->{$contact_obj_1x_key}->{'__I2CONVERT_TYPE'} = "user";
##########################################
# macros (email, pager, address1..6)
##########################################
if(defined($obj_1x_contact->{'email'})) {
$cfg_obj_2x->{'user'}->{$contact_obj_1x_key}->{'__I2CONVERT_MACROS'}->{'email'} = $obj_1x_contact->{'email'};
}
if(defined($obj_1x_contact->{'pager'})) {
$cfg_obj_2x->{'user'}->{$contact_obj_1x_key}->{'__I2CONVERT_MACROS'}->{'pager'} = $obj_1x_contact->{'pager'};
}
for(my $i=1;$i<=6;$i++) {
my $address = "address$i";
if(defined($obj_1x_contact->{$address})) {
$cfg_obj_2x->{'user'}->{$contact_obj_1x_key}->{'__I2CONVERT_MACROS'}->{$address} = $obj_1x_contact->{$address};
}
}
##########################################
# notification_options (service only XXX)
##########################################
if(defined($obj_1x_contact->{'service_notification_options'})) {
$cfg_obj_2x->{'user'}->{$contact_obj_1x_key}->{'__I2CONVERT_NOTIFICATION_FILTERS'} = convert_notification_options_to_filter($obj_1x_contact->{'service_notification_options'});
}
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
####################################################
# migrate renamed attributes
####################################################
$cfg_obj_2x->{'user'}->{$contact_obj_1x_key}->{'user_name'} = $obj_1x_contact->{'contact_name'};
if(defined($obj_1x_contact->{'alias'})) {
$cfg_obj_2x->{'user'}->{$contact_obj_1x_key}->{'display_name'} = escape_str($obj_1x_contact->{'alias'});
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
}
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
delete($cfg_obj_2x->{'user'}->{$contact_obj_1x_key}->{'usergroups'});
if(defined($obj_1x_contact->{'contactgroups'})) {
# check if there's additive inheritance required, and save a flag
if ($obj_1x_contact->{'contactgroups'} =~ /^\+/) {
$cfg_obj_2x->{'user'}->{$contact_obj_1x_key}->{'__I2_CONVERT_UG_ADD'} = 1;
$obj_1x_contact->{'contactgroups'} =~ s/^\+//;
}
push @{$cfg_obj_2x->{'user'}->{$contact_obj_1x_key}->{'usergroups'}}, str2arr_by_delim_without_excludes($obj_1x_contact->{'contactgroups'}, ',', 1);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
#print "DEBUG: usergroups " . join (" ", @{$cfg_obj_2x->{'user'}->{$contact_obj_1x_key}->{'usergroups'}});
}
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# we need to rebuild that notification logic entirely for 2.x
# do that later when all objects are processed and prepared (all relations?)
#say Dumper($notification_commands_2x);
$cfg_obj_2x->{'user'}->{$contact_obj_1x_key}->{'__I2CONVERT_NOTIFICATION_COMMANDS'} = $notification_commands_2x;
}
SKIP_CONTACTS:
######################################
# GROUPS
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
######################################
if (!@$cfg_obj_1x{'hostgroup'}) {
goto SKIP_HOSTGROUPS;
}
# host->hostgroups and hostgroup-members relinked together
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
foreach my $hostgroup_obj_1x_key (keys %{@$cfg_obj_1x{'hostgroup'}}) {
my $obj_1x_hostgroup = @$cfg_obj_1x{'hostgroup'}->{$hostgroup_obj_1x_key};
# clone it into our hash
$cfg_obj_2x->{'hostgroup'}->{$hostgroup_obj_1x_key} = dclone(@$cfg_obj_1x{'hostgroup'}->{$hostgroup_obj_1x_key});
if(defined($obj_1x_hostgroup->{'alias'})) {
$cfg_obj_2x->{'hostgroup'}->{$hostgroup_obj_1x_key}->{'display_name'} = escape_str($obj_1x_hostgroup->{'alias'});
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
}
####################################################
# check if host_groupname exists, if not, try to copy it from 'name' (no template inheritance possible in groups!)
####################################################
if(!defined($obj_1x_hostgroup->{'hostgroup_name'})) {
if(defined($obj_1x_hostgroup->{'name'})) {
$cfg_obj_2x->{'hostgroup'}->{$hostgroup_obj_1x_key}->{'hostgroup_name'} = $obj_1x_hostgroup->{'name'};
}
}
####################################################
# migrate custom attributes
####################################################
$cfg_obj_2x->{'hostgroup'}->{$hostgroup_obj_1x_key}->{'__I2CONVERT_CUSTOM_ATTR'} = obj_convert_custom_attr_to_custom_hash($cfg_obj_2x->{'hostgroup'}->{$hostgroup_obj_1x_key});
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
####################################################
# check if there are members defined, we must re-link them in their host object again
####################################################
if(defined($obj_1x_hostgroup->{'members'})) {
my @hg_members = ();
# check if members is a wildcard, or a possible comma seperated list
# using object tricks - http://docs.icinga.org/latest/en/objecttricks.html#objecttricks-service
# XXX better create a master template where all hosts inherit from, and use additive hostgroups attribute
if ($obj_1x_hostgroup->{'members'} =~ /^\*$/) {
@hg_members = obj_1x_get_all_hostnames_arr($cfg_obj_2x);
} else {
@hg_members = str2arr_by_delim_without_excludes($obj_1x_hostgroup->{'members'}, ',', 1);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
}
foreach my $hg_member (@hg_members) {
my $obj_2x_hg_member = obj_get_host_obj_by_host_name($cfg_obj_2x, $hg_member);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
#print "DEBUG: $hg_member found.\n";
push @{$obj_2x_hg_member->{'hostgroups'}}, $obj_1x_hostgroup->{'hostgroup_name'};
}
}
}
SKIP_HOSTGROUPS:
if (!@$cfg_obj_1x{'servicegroup'}) {
goto SKIP_SERVICEGROUPS;
}
# service->servicegroups and servicegroup->members relinked together
foreach my $servicegroup_obj_1x_key (keys %{@$cfg_obj_1x{'servicegroup'}}) {
my $obj_1x_servicegroup = @$cfg_obj_1x{'servicegroup'}->{$servicegroup_obj_1x_key};
# clone it into our hash
$cfg_obj_2x->{'servicegroup'}->{$servicegroup_obj_1x_key} = dclone(@$cfg_obj_1x{'servicegroup'}->{$servicegroup_obj_1x_key});
if(defined($obj_1x_servicegroup->{'alias'})) {
$cfg_obj_2x->{'servicegroup'}->{$servicegroup_obj_1x_key}->{'display_name'} = escape_str($obj_1x_servicegroup->{'alias'});
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
}
####################################################
# check if service_groupname exists, if not, try to copy it from 'name' (no template inheritance possible in groups!)
####################################################
if(!defined($obj_1x_servicegroup->{'servicegroup_name'})) {
if(defined($obj_1x_servicegroup->{'name'})) {
$cfg_obj_2x->{'servicegroup'}->{$servicegroup_obj_1x_key}->{'servicegroup_name'} = $obj_1x_servicegroup->{'name'};
}
}
####################################################
# migrate custom attributes
####################################################
$cfg_obj_2x->{'servicegroup'}->{$servicegroup_obj_1x_key}->{'__I2CONVERT_CUSTOM_ATTR'} = obj_convert_custom_attr_to_custom_hash($cfg_obj_2x->{'servicegroup'}->{$servicegroup_obj_1x_key});
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
####################################################
# check if there are members defined, we must re-link them in their service object again
####################################################
if(defined($obj_1x_servicegroup->{'members'})) {
# host1,svc1,host2,svc2 is just an insane way of parsing stuff - do NOT sort here.
my @sg_members = str2arr_by_delim_without_excludes($obj_1x_servicegroup->{'members'}, ',', 0);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# just some safety for debugging
if(@sg_members % 2 != 0) {
debug("servicegroup $obj_1x_servicegroup->{'servicegroup_name'} members list not even: $obj_1x_servicegroup->{'members'}");
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
}
#print "DEBUG: $obj_1x_servicegroup->{'servicegroup_name'}: @sg_members\n";
my $obj_2x_sg_member;
while (scalar(@sg_members) > 0) {
my $sg_member_host = shift(@sg_members);
my $sg_member_service = shift(@sg_members);
#print "DEBUG: Looking for $obj_1x_servicegroup->{'servicegroup_name'}: $sg_member_host/$sg_member_service\n";
# since we require the previously looked up unique hostname/service_description, we use the new values in 2x objects (__I2CONVERT_...)
my $obj_2x_sg_member = obj_get_service_obj_by_host_name_service_description($cfg_obj_2x, "__I2CONVERT_SERVICE_HOSTNAME", "__I2CONVERT_SERVICEDESCRIPTION", $sg_member_host, $sg_member_service);
#print "DEBUG: $sg_member_host,$sg_member_service found.\n";
push @{$obj_2x_sg_member->{'servicegroups'}}, $obj_1x_servicegroup->{'servicegroup_name'};
}
#say Dumper($cfg_obj_2x->{'service'});
}
}
SKIP_SERVICEGROUPS:
if (!@$cfg_obj_1x{'contactgroup'}) {
goto SKIP_CONTACTGROUPS;
}
# contact->contactgroups and contactgroup->members relinked together
foreach my $contactgroup_obj_1x_key (keys %{@$cfg_obj_1x{'contactgroup'}}) {
my $obj_1x_contactgroup = @$cfg_obj_1x{'contactgroup'}->{$contactgroup_obj_1x_key};
# clone it into our hash
$cfg_obj_2x->{'usergroup'}->{$contactgroup_obj_1x_key} = dclone(@$cfg_obj_1x{'contactgroup'}->{$contactgroup_obj_1x_key});
$cfg_obj_2x->{'usergroup'}->{$contactgroup_obj_1x_key}->{'__I2CONVERT_TYPE'} = "usergroup";
####################################################
# migrate renamed attributes
####################################################
$cfg_obj_2x->{'usergroup'}->{$contactgroup_obj_1x_key}->{'usergroup_name'} = $obj_1x_contactgroup->{'contactgroup_name'};
if(defined($obj_1x_contactgroup->{'alias'})) {
$cfg_obj_2x->{'usergroup'}->{$contactgroup_obj_1x_key}->{'display_name'} = escape_str($obj_1x_contactgroup->{'alias'});
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
}
####################################################
# check if contact_groupname exists, if not, try to copy it from 'name' (no template inheritance possible in groups!)
####################################################
if(!defined($obj_1x_contactgroup->{'contactgroup_name'})) {
if(defined($obj_1x_contactgroup->{'name'})) {
$cfg_obj_2x->{'usergroup'}->{$contactgroup_obj_1x_key}->{'usergroup_name'} = $obj_1x_contactgroup->{'name'};
}
}
####################################################
# migrate custom attributes
####################################################
$cfg_obj_2x->{'usergroup'}->{$contactgroup_obj_1x_key}->{'__I2CONVERT_CUSTOM_ATTR'} = obj_convert_custom_attr_to_custom_hash($cfg_obj_2x->{'usergroup'}->{$contactgroup_obj_1x_key});
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
####################################################
# check if there are members defined, we must re-link them in their host object again
####################################################
if(defined($obj_1x_contactgroup->{'members'})) {
my @cg_members = str2arr_by_delim_without_excludes($obj_1x_contactgroup->{'members'}, ',', 1);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
foreach my $cg_member (@cg_members) {
my $obj_2x_cg_member = obj_get_contact_obj_by_contact_name($cfg_obj_2x, "user", "user_name", $cg_member);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
#print "DEBUG: $cg_member found.\n";
push @{$obj_2x_cg_member->{'usergroups'}}, $obj_1x_contactgroup->{'contactgroup_name'};
}
}
}
SKIP_CONTACTGROUPS:
######################################
# TIMEPERIODS
######################################
if (!@$cfg_obj_1x{'timeperiod'}) {
goto SKIP_TIMEPERIODS;
}
foreach my $timeperiod_obj_1x_key (keys %{@$cfg_obj_1x{'timeperiod'}}) {
my $obj_1x_timeperiod = @$cfg_obj_1x{'timeperiod'}->{$timeperiod_obj_1x_key};
# clone it into our hash
$cfg_obj_2x->{'timeperiod'}->{$timeperiod_obj_1x_key} = dclone(@$cfg_obj_1x{'timeperiod'}->{$timeperiod_obj_1x_key});
####################################################
# add dependency to ITL template to objects
####################################################
if(defined($icinga2_cfg->{'itl'}->{'timeperiod-template'}) && $icinga2_cfg->{'itl'}->{'timeperiod-template'} ne "") {
push @{$cfg_obj_2x->{'timeperiod'}->{$timeperiod_obj_1x_key}->{'__I2CONVERT_TEMPLATE_NAMES'}}, $icinga2_cfg->{'itl'}->{'timeperiod-template'};
$cfg_obj_2x->{'timeperiod'}->{$timeperiod_obj_1x_key}->{'__I2CONVERT_USES_TEMPLATE'} = 1;
}
####################################################
# display_name -> alias mapping
####################################################
# if there was a timeperiod alias defined, make this the primary display_name for 2x
if(defined($obj_1x_timeperiod->{'alias'})) {
$cfg_obj_2x->{'timeperiod'}->{$timeperiod_obj_1x_key}->{'display_name'} = escape_str($obj_1x_timeperiod->{'alias'});
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
delete($cfg_obj_2x->{'timeperiod'}->{$timeperiod_obj_1x_key}->{'alias'});
}
}
SKIP_TIMEPERIODS:
######################################
# DEPENDENCIES
######################################
if (!@$cfg_obj_1x{'hostdependency'}) {
goto SKIP_HOSTDEPS;
}
foreach my $hostdependency_obj_1x_key (keys %{@$cfg_obj_1x{'hostdependency'}}) {
my $obj_1x_hostdependency = @$cfg_obj_1x{'hostdependency'}->{$hostdependency_obj_1x_key};
# clone it into our hash
$cfg_obj_2x->{'hostdependency'}->{$hostdependency_obj_1x_key} = dclone(@$cfg_obj_1x{'hostdependency'}->{$hostdependency_obj_1x_key});
# 1. the single host_name entries
# host_name is the master host (comma seperated list)
# dependent_host_name is the child host (comma seperated list)
my @master_host_names = str2arr_by_delim_without_excludes($obj_1x_hostdependency->{'host_name'}, ',', 1);
my @child_host_names = str2arr_by_delim_without_excludes($obj_1x_hostdependency->{'dependent_host_name'}, ',', 1);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# go through all child hosts, and push to the parents array
foreach my $child_host_name (@child_host_names) {
my $child_host_obj = obj_get_host_obj_by_host_name($cfg_obj_2x, $child_host_name);
push @{$child_host_obj->{'__I2CONVERT_PARENT_HOSTNAMES'}}, @master_host_names;
}
# 2. the infamous group logic - let's loop because we're cool
my @master_hostgroup_names = str2arr_by_delim_without_excludes($obj_1x_hostdependency->{'hostgroup_name'}, ',', 1);
my @child_hostgroup_names = str2arr_by_delim_without_excludes($obj_1x_hostdependency->{'dependent_hostgroup_name'}, ',', 1);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
my @all_master_hostgroup_hostnames = ();
# get all hosts as array for the master host groups
foreach my $master_hostgroup_name (@master_hostgroup_names) {
my @host_master_hostgroup_hostnames = obj_get_hostnames_arr_by_hostgroup_name($cfg_obj_2x, $master_hostgroup_name);
push @all_master_hostgroup_hostnames, @host_master_hostgroup_hostnames;
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
}
# go through all child hostgroups and fetch their host objects, setting
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
foreach my $child_hostgroup_name (@child_hostgroup_names) {
my @host_child_hostgroup_hostnames = obj_get_hostnames_arr_by_hostgroup_name($cfg_obj_2x, $child_hostgroup_name);
foreach my $host_child_hostgroup_hostname (@host_child_hostgroup_hostnames) {
my $child_host_obj = obj_get_host_obj_by_host_name($cfg_obj_2x, $host_child_hostgroup_hostname);
push @{$child_host_obj->{'__I2CONVERT_PARENT_HOSTNAMES'}}, @all_master_hostgroup_hostnames;
}
}
}
# XXX ugly but works
SKIP_HOSTDEPS:
if (!@$cfg_obj_1x{'servicedependency'}) {
goto SKIP_SVCDEPS;
}
foreach my $servicedependency_obj_1x_key (keys %{@$cfg_obj_1x{'servicedependency'}}) {
my $obj_1x_servicedependency = @$cfg_obj_1x{'servicedependency'}->{$servicedependency_obj_1x_key};
# clone it into our hash
$cfg_obj_2x->{'servicedependency'}->{$servicedependency_obj_1x_key} = dclone(@$cfg_obj_1x{'servicedependency'}->{$servicedependency_obj_1x_key});
# 1. the single host_name / service_description entries
# service_description is a string, while the host_name directive is still a comma seperated list
my @master_host_names = str2arr_by_delim_without_excludes($obj_1x_servicedependency->{'host_name'}, ',', 1);
my @child_host_names = str2arr_by_delim_without_excludes($obj_1x_servicedependency->{'dependent_host_name'}, ',', 1);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
my $master_service_description = $obj_1x_servicedependency->{'service_description'};
my $child_service_description = $obj_1x_servicedependency->{'dependent_service_description'};
# XXX object tricks allow more here
# - comma seperated list of service descriptions on a single *host_name
# - wildcard * for all services on a single *host_name
# go through all child hosts, and get the service object by host_name and our single service_description
foreach my $child_host_name (@child_host_names) {
my $child_service_obj = obj_get_service_obj_by_host_name_service_description($cfg_obj_2x, "__I2CONVERT_SERVICE_HOSTNAME", "__I2CONVERT_SERVICEDESCRIPTION", $child_host_name, $child_service_description);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# stash all master dependencies onto the child service
foreach my $master_host_name (@master_host_names) {
# use some calculated unique key here (no, i will not split the string later! we are perl, we can do hashes)
my $master_key = $master_host_name."-".$master_service_description;
$child_service_obj->{'__I2CONVERT_PARENT_SERVICES'}->{$master_key}->{'host'} = $master_host_name;
$child_service_obj->{'__I2CONVERT_PARENT_SERVICES'}->{$master_key}->{'service'} = $master_service_description;
}
}
# 2. the infamous group logic - but only for hostgroups here
my @master_hostgroup_names = str2arr_by_delim_without_excludes($obj_1x_servicedependency->{'hostgroup_name'}, ',', 1);
my @child_hostgroup_names = str2arr_by_delim_without_excludes($obj_1x_servicedependency->{'dependent_hostgroup_name'}, ',', 1);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
my @all_master_hostgroup_hostnames = ();
# get all hosts as array for the master host groups
foreach my $master_hostgroup_name (@master_hostgroup_names) {
my @host_master_hostgroup_hostnames = obj_get_hostnames_arr_by_hostgroup_name($cfg_obj_2x, $master_hostgroup_name);
push @all_master_hostgroup_hostnames, @host_master_hostgroup_hostnames;
}
#say Dumper($obj_1x_servicedependency);
#say " DEBUG: all master hg hostnames: ".Dumper(@all_master_hostgroup_hostnames);
# go through all child hostgroups and fetch their host objects, setting
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
foreach my $child_hostgroup_name (@child_hostgroup_names) {
my @host_child_hostgroup_hostnames = obj_get_hostnames_arr_by_hostgroup_name($cfg_obj_2x, $child_hostgroup_name); # child hostgroup members
#say " DEBUG: child hg hostnames: ".Dumper(@host_child_hostgroup_hostnames);
foreach my $host_child_hostgroup_hostname (@host_child_hostgroup_hostnames) {
my $child_service_obj = obj_get_service_obj_by_host_name_service_description($cfg_obj_2x, "__I2CONVERT_SERVICE_HOSTNAME", "__I2CONVERT_SERVICEDESCRIPTION", $host_child_hostgroup_hostname, $child_service_description);
# now loop through all master hostgroups and get their hosts
foreach my $master_hostgroup_name (@master_hostgroup_names) {
my @host_master_hostgroup_names = obj_get_hostnames_arr_by_hostgroup_name($cfg_obj_2x, $master_hostgroup_name); # master hostgroup members
foreach my $host_master_hostgroup_hostname (@host_master_hostgroup_names) {
# use some calculated unique key here (no, i will not split the string later! we are perl, we can do hashes)
my $master_key = $host_master_hostgroup_hostname."-".$master_service_description;
$child_service_obj->{'__I2CONVERT_PARENT_SERVICES'}->{$master_key}->{'host'} = $host_master_hostgroup_hostname; # XXX 5th foreach. awesome!
$child_service_obj->{'__I2CONVERT_PARENT_SERVICES'}->{$master_key}->{'service'} = $master_service_description;
}
}
}
}
}
# XXX ugly but works
SKIP_SVCDEPS:
######################################
# SERVICE->HG<-HOSTMEMBERS MAGIC
# we've skipped services without
# host_name before, now deal with them
# hostgroups have been prepared with
# all their members too (!!)
# we're working on 2.x objects now
######################################
# get the max key for hosts (required for adding more)
my $obj_2x_hosts_cnt = (reverse sort {$a <=> $b} (keys %{@$cfg_obj_2x{'host'}}))[0];
#print "FOO: $obj_2x_hosts_cnt\n";
my $obj_2x_services_hg = {};
# filter all services with a hostgroup_name into smaller list
foreach my $service_obj_2x_key (keys %{@$cfg_obj_2x{'service'}}) {
my $obj_2x_service = @$cfg_obj_2x{'service'}->{$service_obj_2x_key};
#print "DEBUG: now checking $obj_2x_service->{'service_description'}...\n";
# skip all services which already got a host_name? which one wins here? XXX
# skip all services without hostgroup_name
next if(!defined($obj_2x_service->{'hostgroup_name'}));
# XXX object tricks allow to use a comma seperated list of hostgroup_names!
# http://docs.icinga.org/latest/en/objecttricks.html
my @hostgroup_names = str2arr_by_delim_without_excludes($obj_2x_service->{'hostgroup_name'}, ',', 1);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
foreach my $hostgroup_name (@hostgroup_names) {
# we need to save all services first, but our new key is the hostgroupname
# so that we can create multiple services for a single hosthg template later on
push @{$obj_2x_services_hg->{$hostgroup_name}}, $service_obj_2x_key;
}
}
# now loop over all hostgroups with service relations
foreach my $service_hg_obj_2x_key (keys %{$obj_2x_services_hg}) {
#say Dumper($obj_2x_services_hg);
# get the stored unique key to our services
my $hg_name = $service_hg_obj_2x_key;
my @service_keys = @{$obj_2x_services_hg->{$hg_name}};
#print "DEBUG: Looking for $hg_name ...\n";
my $obj_2x_hostgroup = obj_get_hostgroup_obj_by_hostgroup_name($cfg_obj_2x, $hg_name);
if(!defined($obj_2x_hostgroup)) {
# no hostgroup defined?
}
# we now need all host names for this hostgroup name, as an array
my @service_hostgroup_hostnames = obj_get_hostnames_arr_by_hostgroup_name($cfg_obj_2x, $hg_name);
if(@service_hostgroup_hostnames == 0) {
# no members, so service cannot be linked. log a warning XXX
#print " DEBUG: no members found, skipping $hg_name\n";
next;
}
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# we've got:
# * n services linked to hostgroups,
# * a hostgroup
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# * an array of hosts as hostgroup members
# we'll create:
# * n service templates,
# * a hg-host template referencing the service templates,
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# * host objects inheriting from it
#
my $svc_count = 0;
# create a host template with hgname-group
my $obj_2x_host_template;
$obj_2x_host_template->{'__I2CONVERT_IS_TEMPLATE'} = 1;
$obj_2x_host_template->{'__I2CONVERT_TEMPLATE_NAME'} = $obj_2x_hostgroup->{'hostgroup_name'}."-group"; # XXX hardcode it for now
# loop through all services and attach them to the host template
foreach my $service_obj_2x_key_val (@service_keys) {
#print "DEBUG: Working on $service_obj_2x_key_val ...\n";
# get the service object by key
my $obj_2x_service = @$cfg_obj_2x{'service'}->{$service_obj_2x_key_val};
# set the service as template.
$obj_2x_service->{'__I2CONVERT_IS_TEMPLATE'} = 1;
$obj_2x_service->{'__I2CONVERT_TEMPLATE_NAME'} = $obj_2x_service->{'service_description'}."-group-".$svc_count; # XXX hardcode it for now
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# create a dummy service inheriting the service template
my $obj_2x_service_inherit;
$obj_2x_service_inherit->{__I2CONVERT_USES_TEMPLATE} = 1;
push @{$obj_2x_service_inherit->{'__I2CONVERT_TEMPLATE_NAMES'}}, $obj_2x_service->{'__I2CONVERT_TEMPLATE_NAME'};
$obj_2x_service_inherit->{'service_description'} = $obj_2x_service->{'service_description'};
$obj_2x_service_inherit->{'__I2CONVERT_SERVICEDESCRIPTION'} = $obj_2x_service->{'service_description'};
# link the service inherit to the host template
$obj_2x_host_template->{'SERVICE'}->{$svc_count} = $obj_2x_service_inherit;
$svc_count++;
}
# all host objects on the hostgroup members will get the host hg template name pushed into their array
foreach my $hostgroup_member_host_name (@service_hostgroup_hostnames) {
# get the host obj
my $obj_2x_host = obj_get_host_obj_by_host_name($cfg_obj_2x, $hostgroup_member_host_name); # this is a reference in memory, not a copy!
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# push the template used
# (override __I2CONVERT_USES_TEMPLATE too)
$obj_2x_host->{__I2CONVERT_USES_TEMPLATE} = 1;
push @{$obj_2x_host->{'__I2CONVERT_TEMPLATE_NAMES'}}, $obj_2x_host_template->{'__I2CONVERT_TEMPLATE_NAME'};
}
# push back the newly created host template (incl the service inherit below SERVICE) to the objects 2.x hive
#say Dumper($obj_2x_host_template);
$obj_2x_hosts_cnt++;
#print "adding new host at key " . $obj_2x_hosts_cnt . "\n";
$cfg_obj_2x->{'host'}->{$obj_2x_hosts_cnt} = $obj_2x_host_template;
}
######################################
# NEW: NOTIFICATION MAPPING
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# old: contact->notification_commands->commands
# contact->email/etc
# host/service -> contact
# new: notification->notification_command
# user->mail/etc
# host/service->notifications[type]->notification_templates,users
######################################
my $notification_obj_cnt = 0;
my $obj_notification_cnt = 0;
# add a dummy value so that we can check against it
$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_NOTIFICATION_NAME'} = '__I2CONVERT_NOTIFICATION_DUMMY';
# go through all users and build notifications based on the notification_command
foreach my $user_obj_2x_key (keys %{@$cfg_obj_2x{'user'}}) {
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
my $obj_2x_user = @$cfg_obj_2x{'user'}->{$user_obj_2x_key};
my $user_notification;
####################################################
# get all notification_commands, and create new notification templates
####################################################
my $notification_commands = $obj_2x_user->{'__I2CONVERT_NOTIFICATION_COMMANDS'};
#say Dumper($notification_commands);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
foreach my $notification_command_type (keys %{$notification_commands}) {
foreach my $notification_command_name (keys %{$notification_commands->{$notification_command_type}}) {
my $notification_command_line = $notification_commands->{$notification_command_type}->{$notification_command_name};
#print "type: $notification_command_type name: $notification_command_name line: $notification_command_line\n";
my $notification_command_name_2x = $notification_command_type."-".$notification_command_name;
my $notification_name_2x = $notification_command_name_2x.$obj_notification_cnt;
$obj_notification_cnt++;
# save a relation to this user and which notification templates are now linked ( ["name"] = { templates = "template" } )
# we'll use that later on when processing hosts/services and linking to users and notifications
$user_notification->{$notification_name_2x}->{'name'} = $notification_name_2x;
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
push @{$user_notification->{$notification_name_2x}->{'templates'}}, $notification_command_name_2x;
push @{$user_notification->{$notification_name_2x}->{'users'}}, $obj_2x_user->{'user_name'};
# save the type for later objects (host or service)
$user_notification->{$notification_name_2x}->{'type'} = $notification_command_type;
# XXX do not add duplicate notifications, they must remain unique by their notification_command origin!
#say Dumper("checking existing $notification_command_name_2x");
#say Dumper($user_notification);
if (obj_2x_notification_exists($cfg_obj_2x, $notification_command_name_2x) == 1) {
#say Dumper("already existing $notification_command_name_2x");
next;
}
next if (!defined($notification_command_name_2x));
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# create a new NotificationCommand 2x object with the original name
my $command_type = 'Notification';
if (obj_2x_command_exists($cfg_obj_2x, $notification_command_name, $command_type) != 1) {
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_TYPE'} = 'Notification';
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_NAME'} = $notification_command_name;
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_LINE'} = $notification_command_line;
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_MACROS'} = $notification_commands->{$notification_command_type}->{'command_macros'};
# use the ITL plugin notification command template
if(defined($icinga2_cfg->{'itl'}->{'notificationcommand-template'}) && $icinga2_cfg->{'itl'}->{'notificationcommand-template'} ne "") {
push @{$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_TEMPLATE_NAMES'}}, $icinga2_cfg->{'itl'}->{'notificationcommand-template'};
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_USES_TEMPLATE'} = 1;
}
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# the check command name of 1.x is still the unique command object name, so we just keep it
# in __I2CONVERT_NOTIFICATION_COMMAND
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# our global PK
$command_obj_cnt++;
}
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# create a new notification template object
$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_NOTIFICATION_TEMPLATE_NAME'} = $notification_command_name_2x;
$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_NOTIFICATION_OBJECT_NAME'} = $notification_command_name_2x;
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_NOTIFICATION_COMMAND'} = $notification_command_name;
$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_IS_TEMPLATE'} = 1; # this is a template, used in hosts/services then
# more reference
$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'users'} = $user_notification->{$notification_name_2x}->{'users'};
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# add dependency to ITL template to objects
if(defined($icinga2_cfg->{'itl'}->{'notification-template'}) && $icinga2_cfg->{'itl'}->{'notification-template'} ne "") {
@{$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_TEMPLATE_NAMES'}} = ();
push @{$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_TEMPLATE_NAMES'}}, $icinga2_cfg->{'itl'}->{'notification-template'};
$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_USES_TEMPLATE'} = 1; # we now use a template, otherwise it won't be dumped
}
$notification_obj_cnt++;
}
}
$cfg_obj_2x->{'user'}->{$user_obj_2x_key}->{'__I2CONVERT_NOTIFICATIONS'} = $user_notification;
#say Dumper($cfg_obj_2x->{'user'}->{$user_obj_2x_key});
}
# go through all hosts/services, and add notifications based on the users
# XXX hosts - do we notify on hosts?
foreach my $host_obj_2x_key (keys %{@$cfg_obj_2x{'host'}}) {
my $obj_2x_host = @$cfg_obj_2x{'host'}->{$host_obj_2x_key};
# make sure there are none
delete($cfg_obj_2x->{'host'}->{$host_obj_2x_key}->{'__I2CONVERT_NOTIFICATIONS'});
@{$cfg_obj_2x->{'host'}->{$host_obj_2x_key}->{'__I2CONVERT_NOTIFICATIONS'}} = ();
# convert users and usergroupmembers into a unique list of users
my @users = str2arr_by_delim_without_excludes($obj_2x_host->{'contacts'}, ',', 1);
my @usergroups = str2arr_by_delim_without_excludes($obj_2x_host->{'contact_groups'}, ',', 1);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# get all members of the usergroups
foreach my $usergroup (@usergroups) {
my @users_ug = obj_get_usernames_arr_by_usergroup_name($cfg_obj_2x, $usergroup);
push @users, @users_ug;
}
# create a unique array of users (XXX important! XXX)
my @uniq_users = uniq(@users);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# now loop and fetch objects, and their needed notification values as array
# (prepared above - look for $user_notification->{$notification_command_name_2x}...)
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
foreach my $uniq_user (@uniq_users) {
my $obj_2x_user = obj_get_user_obj_by_user_name($cfg_obj_2x, $uniq_user);
push @{$cfg_obj_2x->{'host'}->{$host_obj_2x_key}->{'__I2CONVERT_NOTIFICATIONS'}}, $obj_2x_user->{'__I2CONVERT_NOTIFICATIONS'};
# we'll add a reference to all notifications here. decide on dump which object type is given, and dump only those notifications!
}
#say Dumper($obj_2x_service);
}
# XXX services
foreach my $service_obj_2x_key (keys %{@$cfg_obj_2x{'service'}}) {
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
my $obj_2x_service = @$cfg_obj_2x{'service'}->{$service_obj_2x_key};
# make sure there are none
delete($cfg_obj_2x->{'service'}->{$service_obj_2x_key}->{'__I2CONVERT_NOTIFICATIONS'});
@{$cfg_obj_2x->{'service'}->{$service_obj_2x_key}->{'__I2CONVERT_NOTIFICATIONS'}} = ();
# convert users and usergroupmembers into a unique list of users
my @users = str2arr_by_delim_without_excludes($obj_2x_service->{'contacts'}, ',', 1);
my @usergroups = str2arr_by_delim_without_excludes($obj_2x_service->{'contact_groups'}, ',', 1);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# get all members of the usergroups
foreach my $usergroup (@usergroups) {
my @users_ug = obj_get_usernames_arr_by_usergroup_name($cfg_obj_2x, $usergroup);
push @users, @users_ug;
}
# create a unique array of users (XXX important! XXX)
my @uniq_users = uniq(@users);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# now loop and fetch objects, and their needed notification values as array
# (prepared above - look for $user_notification->{$notification_command_name_2x}...)
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
foreach my $uniq_user (@uniq_users) {
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
my $obj_2x_user = obj_get_user_obj_by_user_name($cfg_obj_2x, $uniq_user);
#say Dumper($obj_2x_user);
push @{$cfg_obj_2x->{'service'}->{$service_obj_2x_key}->{'__I2CONVERT_NOTIFICATIONS'}}, $obj_2x_user->{'__I2CONVERT_NOTIFICATIONS'};
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# we'll add a reference to all notifications here. decide on dump which object type is given, and dump only those notifications!
}
# inherit notification attributes from service to all newly created notifications
foreach my $notification (@{$cfg_obj_2x->{'service'}->{$service_obj_2x_key}->{'__I2CONVERT_NOTIFICATIONS'}}) {
foreach my $notification_key (keys %{$notification}) {
next if $notification->{$notification_key}->{'type'} ne 'service';
my $notifications_enabled = obj_1x_get_service_attr($cfg_obj_2x, $obj_2x_service, $obj_2x_service->{'__I2CONVERT_SERVICE_HOSTNAME'}, 'notifications_enabled');
my $notification_interval = obj_1x_get_service_attr($cfg_obj_2x, $obj_2x_service, $obj_2x_service->{'__I2CONVERT_SERVICE_HOSTNAME'}, 'notification_internval');
my $notification_period = obj_1x_get_service_attr($cfg_obj_2x, $obj_2x_service, $obj_2x_service->{'__I2CONVERT_SERVICE_HOSTNAME'}, 'notification_period');
$notification->{$notification_key}->{'__I2CONVERT_NOTIFICATION_FILTERS'} = $cfg_obj_2x->{'service'}->{$service_obj_2x_key}->{'__I2CONVERT_NOTIFICATION_FILTERS'};
$notification->{$notification_key}->{'__I2CONVERT_NOTIFICATION_INTERVAL'} = $notification_interval;
$notification->{$notification_key}->{'__I2CONVERT_NOTIFICATION_PERIOD'} = $notification_period;
$notification->{$notification_key}->{'__I2CONVERT_NOTIFICATION_ENABLED'} = $notifications_enabled;
#say Dumper($notification->{$notification_key});
}
}
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
#say Dumper($obj_2x_service);
}
#exit(0);
######################################
# NEW: ESCALATION TO NOTIFICATION
######################################
my $obj_notification_escal_cnt = 0;
if (!@$cfg_obj_1x{'serviceescalation'}) {
goto SKIP_SVCESCAL;
}
foreach my $serviceescalation_obj_1x_key (keys %{@$cfg_obj_1x{'serviceescalation'}}) {
my $obj_1x_serviceescalation = @$cfg_obj_1x{'serviceescalation'}->{$serviceescalation_obj_1x_key};
######################################
# create a unique users list
######################################
# we need to get all notification_commands and create a notification escalation item from that source
my $notification_prefix = 'serviceescalation';
# convert users and usergroupmembers into a unique list of users
my @users = str2arr_by_delim_without_excludes($obj_1x_serviceescalation->{'contacts'}, ',', 1);
my @usergroups = str2arr_by_delim_without_excludes($obj_1x_serviceescalation->{'contact_groups'}, ',', 1);
# get all members of the usergroups
foreach my $usergroup (@usergroups) {
my @users_ug = obj_get_usernames_arr_by_usergroup_name($cfg_obj_2x, $usergroup);
push @users, @users_ug;
}
# create a unique array of users (XXX important! XXX)
my @uniq_users = uniq(@users);
######################################
# link users to this notification
######################################
foreach my $uniq_user (@uniq_users) {
my $obj_2x_user = obj_get_user_obj_by_user_name($cfg_obj_2x, $uniq_user);
#say Dumper($obj_2x_user);
my $user_notification;
my $notification_commands = $obj_2x_user->{'__I2CONVERT_NOTIFICATION_COMMANDS'};
#say Dumper($notification_commands);
foreach my $notification_command_type (keys %{$notification_commands}) {
foreach my $notification_command_name (keys %{$notification_commands->{$notification_command_type}}) {
my $notification_command_line = $notification_commands->{$notification_command_type}->{$notification_command_name};
#print "type: $notification_command_type name: $notification_command_name line: $notification_command_line\n";
my $notification_command_name_2x = $notification_prefix."-".$notification_command_type."-".$notification_command_name;
my $notification_name_2x = $notification_command_name_2x.$obj_notification_escal_cnt;
$obj_notification_escal_cnt++;
# save a relation to this user and which notification templates are now linked ( ["name"] = { templates = "template" } )
# we'll use that later on when processing hosts/services and linking to users and notifications
$user_notification->{$notification_name_2x}->{'name'} = $notification_name_2x;
push @{$user_notification->{$notification_name_2x}->{'templates'}}, $notification_command_name_2x;
push @{$user_notification->{$notification_name_2x}->{'users'}}, $obj_2x_user->{'user_name'};
# save the type for later objects (host or service)
$user_notification->{$notification_name_2x}->{'type'} = $notification_command_type;
######################################
# create a unique services list, and
# link that to the notification
# - host_name/service_description
# - hostgroup_name/service_description
# - servicegroup_name
######################################
my $notification_interval = 60; # assume some default if everything goes wrong
######################################
# get the obj by host_name/service_description
######################################
if (defined($obj_1x_serviceescalation->{'host_name'}) && defined($obj_1x_serviceescalation->{'service_description'})) {
my $serviceescalation_service_obj = obj_get_service_obj_by_host_name_service_description($cfg_obj_2x, "__I2CONVERT_SERVICE_HOSTNAME", "__I2CONVERT_SERVICEDESCRIPTION", $obj_1x_serviceescalation->{'host_name'}, $obj_1x_serviceescalation->{'service_description'});
push @{$serviceescalation_service_obj->{'__I2CONVERT_NOTIFICATIONS'}}, $user_notification;
#say Dumper($serviceescalation_service_obj);
# we need to calculate begin/end based on service->notification_interval
# if notification_interval is not defined, we need to look it up in the template tree!
if (defined($serviceescalation_service_obj->{'notification_interval'})) {
$notification_interval = $serviceescalation_service_obj->{'notification_interval'};
} else {
$notification_interval = obj_1x_get_service_attr($cfg_obj_1x, $serviceescalation_service_obj, $serviceescalation_service_obj->{'__I2CONVERT_SERVICE_HOSTNAME'}, 'notification_interval');
}
#say Dumper($notification_interval);
$user_notification->{$notification_name_2x}->{'__I2CONVERT_NOTIFICATION_TIMES'}->{'begin'} = $obj_1x_serviceescalation->{'first_notification'} * $notification_interval;
$user_notification->{$notification_name_2x}->{'__I2CONVERT_NOTIFICATION_TIMES'}->{'end'} = $obj_1x_serviceescalation->{'last_notification'} * $notification_interval;
# save a reference to more infos
$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_NOTIFICATION_TIMES'} = $user_notification->{$notification_name_2x}->{'__I2CONVERT_NOTIFICATION_TIMES'};
#say Dumper($obj_1x_serviceescalation);
#say Dumper($user_notification);
######################################
# now ADD the new escalation notification
######################################
# XXX do not add duplicate notifications, they must remain unique by their notification_command origin!
next if (obj_2x_notification_exists($cfg_obj_2x, $notification_command_name_2x) == 1);
next if (!defined($notification_command_name_2x));
# create a new NotificationCommand 2x object with the original name
if (obj_2x_notification_exists($cfg_obj_2x, $notification_command_name) != 1) {
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_TYPE'} = 'Notification';
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_NAME'} = $notification_command_name;
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_LINE'} = $notification_command_line;
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_MACROS'} = $notification_commands->{$notification_command_type}->{'command_macros'};
# use the ITL plugin notification command template
if(defined($icinga2_cfg->{'itl'}->{'notificationcommand-template'}) && $icinga2_cfg->{'itl'}->{'notificationcommand-template'} ne "") {
push @{$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_TEMPLATE_NAMES'}}, $icinga2_cfg->{'itl'}->{'notificationcommand-template'};
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_USES_TEMPLATE'} = 1;
}
# the check command name of 1.x is still the unique command object name, so we just keep it
# in __I2CONVERT_NOTIFICATION_COMMAND
# our global PK
$command_obj_cnt++;
}
# create a new notification template object
$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_NOTIFICATION_TEMPLATE_NAME'} = $notification_command_name_2x;
$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_NOTIFICATION_OBJECT_NAME'} = $notification_command_name_2x;
$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_NOTIFICATION_COMMAND'} = $notification_command_name;
$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_IS_TEMPLATE'} = 1; # this is a template, used in hosts/services then
# more reference
$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'users'} = $user_notification->{$notification_name_2x}->{'users'};
# add dependency to ITL template to objects
if(defined($icinga2_cfg->{'itl'}->{'notification-template'}) && $icinga2_cfg->{'itl'}->{'notification-template'} ne "") {
@{$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_TEMPLATE_NAMES'}} = ();
push @{$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_TEMPLATE_NAMES'}}, $icinga2_cfg->{'itl'}->{'notification-template'};
$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_USES_TEMPLATE'} = 1; # we now use a template, otherwise it won't be dumped
}
#say Dumper($cfg_obj_2x->{'notification'}->{$notification_obj_cnt});
# our PK
$notification_obj_cnt++;
}
######################################
# get all hosts in hostgroup, with service_description
######################################
if (defined($obj_1x_serviceescalation->{'hostgroup_name'}) && defined($obj_1x_serviceescalation->{'service_description'})) {
my @serviceescalation_hostgroup_names = str2arr_by_delim_without_excludes($obj_1x_serviceescalation->{'hostgroup_name'}, ',', 1);
foreach my $serviceescalation_hostgroup_name (@serviceescalation_hostgroup_names) {
# get hg members
my @serviceescalation_hostgroup_hostnames = obj_get_hostnames_arr_by_hostgroup_name($cfg_obj_2x, $serviceescalation_hostgroup_name);
foreach my $serviceescalation_hostgroup_hostname (@serviceescalation_hostgroup_hostnames) {
if (defined($obj_1x_serviceescalation->{'service_description'})) {
my $serviceescalation_service_obj = obj_get_service_obj_by_host_name_service_description($cfg_obj_2x, "__I2CONVERT_SERVICE_HOSTNAME", "__I2CONVERT_SERVICEDESCRIPTION", $obj_1x_serviceescalation->{'host_name'}, $obj_1x_serviceescalation->{'service_description'});
push @{$serviceescalation_service_obj->{'__I2CONVERT_NOTIFICATIONS'}}, $user_notification;
# we need to calculate begin/end based on service->notification_interval
# if notification_interval is not defined, we need to look it up in the template tree!
if (defined($serviceescalation_service_obj->{'notification_interval'})) {
$notification_interval = $serviceescalation_service_obj->{'notification_interval'};
} else {
$notification_interval = obj_1x_get_service_attr($cfg_obj_1x, $serviceescalation_service_obj, $serviceescalation_service_obj->{'__I2CONVERT_SERVICE_HOSTNAME'}, 'notification_interval');
}
$user_notification->{$notification_name_2x}->{'__I2CONVERT_NOTIFICATION_TIMES'}->{'begin'} = $obj_1x_serviceescalation->{'first_notification'} * $notification_interval;
$user_notification->{$notification_name_2x}->{'__I2CONVERT_NOTIFICATION_TIMES'}->{'end'} = $obj_1x_serviceescalation->{'last_notification'} * $notification_interval;
# save a reference to more infos
$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_NOTIFICATION_TIMES'} = $user_notification->{$notification_name_2x}->{'__I2CONVERT_NOTIFICATION_TIMES'};
######################################
# now ADD the new escalation notification
######################################
# XXX do not add duplicate notifications, they must remain unique by their notification_command origin!
next if (obj_2x_notification_exists($cfg_obj_2x, $notification_command_name_2x) == 1);
next if (!defined($notification_command_name_2x));
# create a new NotificationCommand 2x object with the original name
if (obj_2x_notification_exists($cfg_obj_2x, $notification_command_name) != 1) {
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_TYPE'} = 'Notification';
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_NAME'} = $notification_command_name;
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_LINE'} = $notification_command_line;
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_MACROS'} = $notification_commands->{$notification_command_type}->{'command_macros'};
# use the ITL plugin notification command template
if(defined($icinga2_cfg->{'itl'}->{'notificationcommand-template'}) && $icinga2_cfg->{'itl'}->{'notificationcommand-template'} ne "") {
push @{$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_TEMPLATE_NAMES'}}, $icinga2_cfg->{'itl'}->{'notificationcommand-template'};
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_USES_TEMPLATE'} = 1;
}
# the check command name of 1.x is still the unique command object name, so we just keep it
# in __I2CONVERT_NOTIFICATION_COMMAND
# our global PK
$command_obj_cnt++;
}
# create a new notification template object
$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_NOTIFICATION_TEMPLATE_NAME'} = $notification_command_name_2x;
$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_NOTIFICATION_OBJECT_NAME'} = $notification_command_name_2x;
$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_NOTIFICATION_COMMAND'} = $notification_command_name;
$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_IS_TEMPLATE'} = 1; # this is a template, used in hosts/services then
# more references (this is why code duplication happens
$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'users'} = $user_notification->{$notification_name_2x}->{'users'};
# add dependency to ITL template to objects
if(defined($icinga2_cfg->{'itl'}->{'notification-template'}) && $icinga2_cfg->{'itl'}->{'notification-template'} ne "") {
@{$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_TEMPLATE_NAMES'}} = ();
push @{$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_TEMPLATE_NAMES'}}, $icinga2_cfg->{'itl'}->{'notification-template'};
$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_USES_TEMPLATE'} = 1; # we now use a template, otherwise it won't be dumped
}
# our PK
$notification_obj_cnt++;
}
}
}
}
#say Dumper($cfg_obj_2x->{'notification'}->{$notification_obj_cnt});
######################################
# get all hosts and services from servicegroup definition and link them
######################################
# XXX FIXME
if (defined($obj_1x_serviceescalation->{'servicegroup_name'})) {
#say Dumper($obj_1x_serviceescalation);
my @service_names = obj_2x_get_service_arr_by_servicegroup_name($cfg_obj_2x, $obj_1x_serviceescalation->{'servicegroup_name'});
foreach my $serviceescalation_service_name (@service_names) {
my $serviceescalation_service_obj = obj_get_service_obj_by_host_name_service_description($cfg_obj_2x, "__I2CONVERT_SERVICE_HOSTNAME", "__I2CONVERT_SERVICEDESCRIPTION", $serviceescalation_service_name->{'__I2CONVERT_SERVICE_HOSTNAME'}, $serviceescalation_service_name->{'__I2CONVERT_SERVICEDESCRIPTION'});
#say Dumper($serviceescalation_service_obj);
# skip any templates which would create duplicates
next if ($serviceescalation_service_obj->{'__I2CONVERT_IS_TEMPLATE'} == 1);
#say Dumper($serviceescalation_service_name);
push @{$serviceescalation_service_obj->{'__I2CONVERT_NOTIFICATIONS'}}, $user_notification;
# we need to calculate begin/end based on service->notification_interval
# if notification_interval is not defined, we need to look it up in the template tree!
if (defined($serviceescalation_service_obj->{'notification_interval'})) {
$notification_interval = $serviceescalation_service_obj->{'notification_interval'};
} else {
$notification_interval = obj_1x_get_service_attr($cfg_obj_1x, $serviceescalation_service_obj, $serviceescalation_service_obj->{'__I2CONVERT_SERVICE_HOSTNAME'}, 'notification_interval');
}
$user_notification->{$notification_name_2x}->{'__I2CONVERT_NOTIFICATION_TIMES'}->{'begin'} = $obj_1x_serviceescalation->{'first_notification'} * $notification_interval;
$user_notification->{$notification_name_2x}->{'__I2CONVERT_NOTIFICATION_TIMES'}->{'end'} = $obj_1x_serviceescalation->{'last_notification'} * $notification_interval;
# save a reference to more infos
$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_NOTIFICATION_TIMES'} = $user_notification->{$notification_name_2x}->{'__I2CONVERT_NOTIFICATION_TIMES'};
######################################
# now ADD the new escalation notification
######################################
# XXX do not add duplicate notifications, they must remain unique by their notification_command origin!
next if (obj_2x_notification_exists($cfg_obj_2x, $notification_command_name_2x) == 1);
next if (!defined($notification_command_name_2x));
# create a new NotificationCommand 2x object with the original name
if (obj_2x_notification_exists($cfg_obj_2x, $notification_command_name) != 1) {
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_TYPE'} = 'Notification';
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_NAME'} = $notification_command_name;
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_LINE'} = $notification_command_line;
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_MACROS'} = $notification_commands->{$notification_command_type}->{'command_macros'};
# use the ITL plugin notification command template
if(defined($icinga2_cfg->{'itl'}->{'notificationcommand-template'}) && $icinga2_cfg->{'itl'}->{'notificationcommand-template'} ne "") {
push @{$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_TEMPLATE_NAMES'}}, $icinga2_cfg->{'itl'}->{'notificationcommand-template'};
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_USES_TEMPLATE'} = 1;
}
# the check command name of 1.x is still the unique command object name, so we just keep it
# in __I2CONVERT_NOTIFICATION_COMMAND
# our global PK
$command_obj_cnt++;
}
# create a new notification template object
$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_NOTIFICATION_TEMPLATE_NAME'} = $notification_command_name_2x;
$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_NOTIFICATION_OBJECT_NAME'} = $notification_command_name_2x;
$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_NOTIFICATION_COMMAND'} = $notification_command_name;
$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_IS_TEMPLATE'} = 1; # this is a template, used in hosts/services then
# more references (this is why code duplication happens
$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'users'} = $user_notification->{$notification_name_2x}->{'users'};
# add dependency to ITL template to objects
if(defined($icinga2_cfg->{'itl'}->{'notification-template'}) && $icinga2_cfg->{'itl'}->{'notification-template'} ne "") {
@{$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_TEMPLATE_NAMES'}} = ();
push @{$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_TEMPLATE_NAMES'}}, $icinga2_cfg->{'itl'}->{'notification-template'};
$cfg_obj_2x->{'notification'}->{$notification_obj_cnt}->{'__I2CONVERT_USES_TEMPLATE'} = 1; # we now use a template, otherwise it won't be dumped
}
# our PK
$notification_obj_cnt++;
}
}
}
}
$obj_2x_user->{'__I2CONVERT_NOTIFICATIONS'} = $user_notification;
}
}
SKIP_SVCESCAL:
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
######################################
# HOST->SERVICE MAGIC
# we need to do it _after_ we've
# manipulated all service objects!
######################################
# save the current service count for additional hostcheck services
my $service_object_cnt = keys %{@$cfg_obj_2x{'service'}};
$count_original_services = $service_object_cnt;
$service_object_cnt++;
debug("HOST->SERVICE RELATION START: service object count " . keys %{@$cfg_obj_2x{'service'}});
# "get all 'host' hashref as array in hashmap, and their keys to access it"
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
foreach my $host_obj_2x_key (keys %{@$cfg_obj_2x{'host'}}) {
#say Dumper(@$cfg_obj_2x{'host'}->{$host_obj_2x_key});
my $obj_2x_host = @$cfg_obj_2x{'host'}->{$host_obj_2x_key};
# if there's no host check_command, we must look it up in the tree
if(!defined($obj_2x_host->{'check_command'})) {
@$cfg_obj_2x{'host'}->{$host_obj_2x_key}->{'check_command'} = obj_1x_get_host_attr($cfg_obj_1x, $obj_2x_host, $obj_2x_host->{'__I2CONVERT_HOSTNAME'}, 'check_command');
#say Dumper("found check_command in ".$obj_2x_host->{'check_command'});
#say Dumper(@$cfg_obj_2x{'host'}->{$host_obj_2x_key});
}
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
####################################################
# Create Host->Service Relation for later dumping
# we use the prep'ed 2x service hashref already
# all attributes _must_have been resolved already!
####################################################
my $obj_2x_host_service_cnt = 0;
# find all services for this host
foreach my $service_obj_2x_key (keys %{@$cfg_obj_2x{'service'}}) {
my $obj_2x_service = @$cfg_obj_2x{'service'}->{$service_obj_2x_key};
######################################
# get host_name/service_desc for obj
# (prepared in service loop already)
######################################
my $obj_2x_service_host_name = $obj_2x_service->{'__I2CONVERT_SERVICE_HOSTNAME'};
my $obj_2x_service_service_description = $obj_2x_service->{'__I2CONVERT_SERVICEDESCRIPTION'};
######################################
# skip service templates
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
######################################
if ($obj_2x_service->{'__I2CONVERT_IS_TEMPLATE'} == 1) {
#debug("WARNING: Skipping service template '$obj_2x_service->{'__I2CONVERT_TEMPLATE_NAMES'}' for linking to host '$obj_2x_host->{'__I2CONVERT_HOSTNAME'}'.");
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
next;
}
# save it for later
# XXX if host_name can't be located in the service template tree, check if hostgroup is set somewhere
# we then need to check if the service -> hostgroup <- hostmember applies (ugly) FIXME
# XXX if host_name can't be determined, log an error XXX templates MUST be skipped before (they cannot look down, only up in use tree)
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
if (!defined($obj_2x_service_host_name)) {
#print "ERROR: No host_name for service given " . Dumper($obj_2x_service);
next;
}
######################################
# found a host->service relation?
######################################
if ($obj_2x_service_host_name eq $obj_2x_host->{'__I2CONVERT_HOSTNAME'}) {
#debug("service_description: $obj_2x_service_service_description host_name: $obj_2x_service_host_name");
# 1. generate template name "host-service"
my $service_template_name = $obj_2x_service_host_name."-".$obj_2x_service_service_description;
# make sure that service (template) object does not exist XXX
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# 2. make the service object a template with a special unique name
$cfg_obj_2x->{'service'}->{$service_obj_2x_key}->{'__I2CONVERT_IS_TEMPLATE'} = 1;
$cfg_obj_2x->{'service'}->{$service_obj_2x_key}->{'__I2CONVERT_TEMPLATE_NAME'} = $service_template_name;
# 3. use the template name as reference for the host->service
$cfg_obj_2x->{'host'}->{$host_obj_2x_key}->{'SERVICE'}->{$obj_2x_host_service_cnt}->{'__I2CONVERT_USES_TEMPLATE'} = 1;
push @{$cfg_obj_2x->{'host'}->{$host_obj_2x_key}->{'SERVICE'}->{$obj_2x_host_service_cnt}->{'__I2CONVERT_TEMPLATE_NAMES'}}, $service_template_name;
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# 4. define the service description for the service
$cfg_obj_2x->{'host'}->{$host_obj_2x_key}->{'SERVICE'}->{$obj_2x_host_service_cnt}->{'__I2CONVERT_SERVICEDESCRIPTION'} = $obj_2x_service_service_description;
# increment the PK now
$obj_2x_host_service_cnt++;
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
######################################
# LINK HOST COMMAND WITH SERVICE CHECK
######################################
my ($service_check_command_2x, @service_command_args_1x) = convert_checkcommand($cfg_obj_1x, @$cfg_obj_1x{'command'}, $obj_2x_service, $obj_2x_service_host_name, $global_macros_1x);
my ($host_check_command_2x, @host_command_args_1x) = convert_checkcommand($cfg_obj_1x, @$cfg_obj_1x{'command'}, $obj_2x_host, $obj_2x_service_host_name, $global_macros_1x);
#say Dumper($host_check_command_2x);
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# check if this service check is a possible match for __I2CONVERT_HOST_CHECK?
if (defined($service_check_command_2x->{'check_command_name_1x'})) {
#say ("====================================");
#say ($service_check_command_2x->{'check_command_name_1x'});
#say ("-------------");
#say ($cfg_obj_2x->{'host'}->{$host_obj_2x_key}->{'__I2CONVERT_HOSTCHECK_NAME'});
#say ("====================================");
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
if ($cfg_obj_2x->{'host'}->{$host_obj_2x_key}->{'__I2CONVERT_HOSTCHECK_NAME'} eq $service_check_command_2x->{'check_command_name_1x'}) {
# set service as hostcheck
$cfg_obj_2x->{'host'}->{$host_obj_2x_key}->{'__I2CONVERT_HOSTCHECK'} = $obj_2x_service_service_description;
} else {
######################################
# we need to take the old host check_command and create a new service object, linked to it
######################################
next if (!defined($host_check_command_2x->{'check_command_name_1x'}));
# XXX do not add duplicate check commands, they must remain unique by their check_command origin!
my $command_type = 'Check';
if (obj_2x_command_exists($cfg_obj_2x, $host_check_command_2x->{'check_command_name_1x'}, $command_type) != 1) {
# create a new CheckCommand 2x object with the original name
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_TYPE'} = 'Check';
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_NAME'} = $host_check_command_2x->{'check_command_name_1x'};
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_LINE'} = $host_check_command_2x->{'check_command'};
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_MACROS'} = $host_check_command_2x->{'command_macros'};
# use the ITL plugin check command template
if(defined($icinga2_cfg->{'itl'}->{'checkcommand-template'}) && $icinga2_cfg->{'itl'}->{'checkcommand-template'} ne "") {
push @{$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_TEMPLATE_NAMES'}}, $icinga2_cfg->{'itl'}->{'checkcommand-template'};
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_USES_TEMPLATE'} = 1;
}
# our PK
$command_obj_cnt++;
}
######################################
# now create a new service template linked to the host
######################################
my $obj_2x_service_service_description = $host_check_command_2x->{'check_command_name_1x'};
if (obj_2x_host_service_exists($cfg_obj_2x->{'host'}->{$host_obj_2x_key}->{'SERVICE'}, $obj_2x_service_service_description) != 1) {
# 1. generate template name "host-service"
my $service_template_name = $obj_2x_service_host_name."-".$obj_2x_service_service_description;
if (obj_2x_hostcheck_service_template_exists($cfg_obj_2x->{'service'}, $service_template_name) != 1) {
# 2. create a new service template object based on $service_object_cnt
$cfg_obj_2x->{'service'}->{$service_object_cnt}->{'__I2CONVERT_IS_TEMPLATE'} = 1;
$cfg_obj_2x->{'service'}->{$service_object_cnt}->{'__I2CONVERT_TEMPLATE_NAME'} = $service_template_name;
$cfg_obj_2x->{'service'}->{$service_object_cnt}->{'__I2CONVERT_SERVICEDESCRIPTION'} = $obj_2x_service_service_description;
$cfg_obj_2x->{'service'}->{$service_object_cnt}->{'__I2_CONVERT_CHECKCOMMAND_NAME'} = $host_check_command_2x->{'check_command_name_1x'};
# 3. save all command args as macros
my $arg_cnt = 1;
foreach my $command_arg_1x (@host_command_args_1x) {
my $obj_1x_host = obj_get_host_obj_by_host_name($cfg_obj_1x, $host_obj_2x_key);
my $command_arg_2x = resolve_macros($cfg_obj_1x, $obj_1x_host, $command_arg_1x);
$cfg_obj_2x->{'service'}->{$service_object_cnt}->{'__I2CONVERT_MACROS'}->{"ARG" . $arg_cnt} = escape_str($command_arg_2x);
$arg_cnt++;
}
# 4. use the ITL service template
if(defined($icinga2_cfg->{'itl'}->{'service-template'}) && $icinga2_cfg->{'itl'}->{'service-template'} ne "") {
push @{$cfg_obj_2x->{'service'}->{$service_object_cnt}->{'__I2CONVERT_TEMPLATE_NAMES'}}, $icinga2_cfg->{'itl'}->{'service-template'};
$cfg_obj_2x->{'service'}->{$service_object_cnt}->{'__I2CONVERT_USES_TEMPLATE'} = 1;
}
# primary service key
$service_object_cnt++;
}
# 5. link the hostcheck service to the template
$cfg_obj_2x->{'host'}->{$host_obj_2x_key}->{'SERVICE'}->{$obj_2x_host_service_cnt}->{'__I2CONVERT_USES_TEMPLATE'} = 1;
push @{$cfg_obj_2x->{'host'}->{$host_obj_2x_key}->{'SERVICE'}->{$obj_2x_host_service_cnt}->{'__I2CONVERT_TEMPLATE_NAMES'}}, $service_template_name;
# 6. set hostservice attributes
$cfg_obj_2x->{'host'}->{$host_obj_2x_key}->{'SERVICE'}->{$obj_2x_host_service_cnt}->{'__I2CONVERT_IS_TEMPLATE'} = 0;
$cfg_obj_2x->{'host'}->{$host_obj_2x_key}->{'SERVICE'}->{$obj_2x_host_service_cnt}->{'__I2CONVERT_SERVICEDESCRIPTION'} = $obj_2x_service_service_description;
# primary key
$obj_2x_host_service_cnt++;
}
######################################
# now link the service desc to hostcheck
######################################
$cfg_obj_2x->{'host'}->{$host_obj_2x_key}->{'__I2CONVERT_HOSTCHECK'} = $obj_2x_service_service_description;
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
}
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
}
}
else {
# no match
#say "ERROR: No Match with ". Dumper($obj_1x_host);
}
}
######################################
# this host object does not have any
# services assigned
######################################
if ($obj_2x_host_service_cnt == 0 && $obj_2x_host->{'__I2CONVERT_IS_TEMPLATE'} == 0) {
debug("Found host '". $obj_2x_host->{'__I2CONVERT_HOSTNAME'} ."' without any services");
# now, add the hostcheck service
my ($host_check_command_2x, @host_command_args_1x) = convert_checkcommand($cfg_obj_1x, @$cfg_obj_1x{'command'}, $obj_2x_host, $obj_2x_host->{'__I2CONVERT_HOSTNAME'} , $global_macros_1x);
next if (!defined($host_check_command_2x->{'check_command_name_1x'}));
my $command_type = 'Check';
if (obj_2x_command_exists($cfg_obj_2x, $host_check_command_2x->{'check_command_name_1x'}, $command_type) != 1) {
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_TYPE'} = 'Check';
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_NAME'} = $host_check_command_2x->{'check_command_name_1x'};
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_LINE'} = $host_check_command_2x->{'check_command'};
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_COMMAND_MACROS'} = $host_check_command_2x->{'command_macros'};
# use the ITL plugin check command template
if(defined($icinga2_cfg->{'itl'}->{'checkcommand-template'}) && $icinga2_cfg->{'itl'}->{'checkcommand-template'} ne "") {
push @{$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_TEMPLATE_NAMES'}}, $icinga2_cfg->{'itl'}->{'checkcommand-template'};
$cfg_obj_2x->{'command'}->{$command_obj_cnt}->{'__I2CONVERT_USES_TEMPLATE'} = 1;
}
# our PK
$command_obj_cnt++;
}
# now create a new service template linked to the host
my $obj_2x_service_service_description = $host_check_command_2x->{'check_command_name_1x'};
my $obj_2x_service_host_name = $obj_2x_host->{'__I2CONVERT_HOSTNAME'};
if (obj_2x_host_service_exists($cfg_obj_2x->{'host'}->{$host_obj_2x_key}->{'SERVICE'}, $obj_2x_service_service_description) != 1) {
# 1. generate template name "host-service"
my $service_template_name = $obj_2x_service_host_name."-".$obj_2x_service_service_description;
if (obj_2x_hostcheck_service_template_exists($cfg_obj_2x->{'service'}, $service_template_name) != 1) {
# 2. create a new service template object based on $service_object_cnt
$cfg_obj_2x->{'service'}->{$service_object_cnt}->{'__I2CONVERT_IS_TEMPLATE'} = 1;
$cfg_obj_2x->{'service'}->{$service_object_cnt}->{'__I2CONVERT_TEMPLATE_NAME'} = $service_template_name;
$cfg_obj_2x->{'service'}->{$service_object_cnt}->{'__I2CONVERT_SERVICEDESCRIPTION'} = $obj_2x_service_service_description;
$cfg_obj_2x->{'service'}->{$service_object_cnt}->{'__I2_CONVERT_CHECKCOMMAND_NAME'} = $host_check_command_2x->{'check_command_name_1x'};
# 3. save all command args as macros
my $arg_cnt = 1;
foreach my $command_arg_1x (@host_command_args_1x) {
my $obj_1x_host = obj_get_host_obj_by_host_name($cfg_obj_1x, $host_obj_2x_key);
my $command_arg_2x = resolve_macros($cfg_obj_1x, $obj_1x_host, $command_arg_1x);
$cfg_obj_2x->{'service'}->{$service_object_cnt}->{'__I2CONVERT_MACROS'}->{"ARG" . $arg_cnt} = escape_str($command_arg_2x);
$arg_cnt++;
}
# 4. use the ITL service template
if(defined($icinga2_cfg->{'itl'}->{'service-template'}) && $icinga2_cfg->{'itl'}->{'service-template'} ne "") {
push @{$cfg_obj_2x->{'service'}->{$service_object_cnt}->{'__I2CONVERT_TEMPLATE_NAMES'}}, $icinga2_cfg->{'itl'}->{'service-template'};
$cfg_obj_2x->{'service'}->{$service_object_cnt}->{'__I2CONVERT_USES_TEMPLATE'} = 1;
}
# primary service key
$service_object_cnt++;
}
# 5. link the hostcheck service to the template
$cfg_obj_2x->{'host'}->{$host_obj_2x_key}->{'SERVICE'}->{$obj_2x_host_service_cnt}->{'__I2CONVERT_USES_TEMPLATE'} = 1;
push @{$cfg_obj_2x->{'host'}->{$host_obj_2x_key}->{'SERVICE'}->{$obj_2x_host_service_cnt}->{'__I2CONVERT_TEMPLATE_NAMES'}}, $service_template_name;
# 6. set hostservice attributes
$cfg_obj_2x->{'host'}->{$host_obj_2x_key}->{'SERVICE'}->{$obj_2x_host_service_cnt}->{'__I2CONVERT_IS_TEMPLATE'} = 0;
$cfg_obj_2x->{'host'}->{$host_obj_2x_key}->{'SERVICE'}->{$obj_2x_host_service_cnt}->{'__I2CONVERT_SERVICEDESCRIPTION'} = $obj_2x_service_service_description;
# primary key
$obj_2x_host_service_cnt++;
}
# now link the service desc to hostcheck
$cfg_obj_2x->{'host'}->{$host_obj_2x_key}->{'__I2CONVERT_HOSTCHECK'} = $obj_2x_service_service_description;
say "Added hostcheck $obj_2x_service_service_description to host $obj_2x_host->{'__I2CONVERT_HOSTNAME'}";
}
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
}
############################################################################
# add custom templates to every existing object
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
############################################################################
my @objects = ('host', 'service', 'user', 'notification', 'timeperiod', 'command');
foreach my $object (@objects) {
debug("Custom template for $object");
#say "Custom template for $object";
foreach my $obj_2x_key (keys %{@$cfg_obj_2x{$object}}) {
my $obj_2x = @$cfg_obj_2x{$object}->{$obj_2x_key};
my $obj_template = "$object-template";
if (defined($obj_2x->{'__I2CONVERT_COMMAND_TYPE'})) {
$obj_template = lc $obj_2x->{'__I2CONVERT_COMMAND_TYPE'} . "$object-template";
}
if(defined($icinga2_cfg->{'customtmpl'}->{$obj_template}) && $icinga2_cfg->{'customtmpl'}->{$obj_template} ne "") {
if (!defined($cfg_obj_2x->{$object}->{$obj_2x_key}->{'__I2CONVERT_TEMPLATE_NAMES'})) {
@{$cfg_obj_2x->{$object}->{$obj_2x_key}->{'__I2CONVERT_TEMPLATE_NAMES'}} = ();
}
push @{$cfg_obj_2x->{$object}->{$obj_2x_key}->{'__I2CONVERT_TEMPLATE_NAMES'}}, $icinga2_cfg->{'customtmpl'}->{$obj_template};
$cfg_obj_2x->{$object}->{$obj_2x_key}->{'__I2CONVERT_USES_TEMPLATE'} = 1;
}
}
}
############################################################################
# stats
############################################################################
$count->{'hosts'} = keys %{@$cfg_obj_2x{'host'}};
#$count->{'host_templates'} = grep $_->{'__I2_CONVERT_IS_TEMPLATE'} == 1, values %{@$cfg_obj_2x{'host'}};
#$count->{'host_objects'} = grep $_->{'__I2_CONVERT_IS_TEMPLATE'} != 1, values %{@$cfg_obj_2x{'host'}};
$count->{'services_hostcheck'} = $service_object_cnt - $count_original_services;
$count->{'services_original'} = $count_original_services;
$count->{'services'} = $service_object_cnt;
#$count->{'service_templates'} = grep $_->{'__I2_CONVERT_IS_TEMPLATE'} == 1, values %{@$cfg_obj_2x{'service'}};
#$count->{'service_objects'} = grep $_->{'__I2_CONVERT_IS_TEMPLATE'} != 1, values %{@$cfg_obj_2x{'service'}};
$count->{'users'} = keys %{@$cfg_obj_2x{'user'}};
$count->{'timeperiods'} = keys %{@$cfg_obj_2x{'timeperiod'}};
$count->{'notifications'} = keys %{@$cfg_obj_2x{'notification'}};
$count->{'commands'} = keys %{@$cfg_obj_2x{'command'}};
$cfg_obj_2x->{'__I2_CONVERT_STATS'} = $count;
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
# export takes place outside again
return $cfg_obj_2x;
}
=pod
EXPORT ICINGA 2 CONFIG
=cut
# XXX figure a better way for \t count printing, making dumps more modular
# XXX sort macros and linked hostservices by name
################################################################################
# Helpers
################################################################################
sub open_cfg_file {
my $file = shift;
my $FH;
say "writing file '$file'...\n";
open($FH, ">".$file);
if (!-w $FH) {
print "ERROR: cannot write file '$file'. Check permissions!\n";
}
return $FH;
}
sub close_cfg_file {
my $FH = shift;
close($FH);
}
sub add_header {
my $icinga2_cfg = shift;
dump_config_line($icinga2_cfg, "/******************************************************************************");
dump_config_line($icinga2_cfg, " * GENERATED BY ICINGA2 CONVERSION SCRIPT");
dump_config_line($icinga2_cfg, " * " . strftime('%Y-%m-%d %H:%M:%S',localtime));
dump_config_line($icinga2_cfg, " ******************************************************************************/");
}
sub dump_config_line {
my $icinga2_cfg = shift;
my $line = shift;
my $dbg_lvl = $icinga2_cfg->{'__I2EXPORT_DEBUG'};
if ($dbg_lvl) {
print $line. "\n";
}
print { $icinga2_cfg->{'__I2EXPORT_FH'} } "$line\n";
}
sub start_object_type_config_dump {
my $icinga2_cfg = shift;
my $cfg_type = shift;
$icinga2_cfg->{'__I2EXPORT_FH'} = open_cfg_file($icinga2_cfg->{$cfg_type});
add_header($icinga2_cfg);
}
sub end_object_type_config_dump {
my $icinga2_cfg = shift;
close_cfg_file($icinga2_cfg->{'__I2EXPORT_FH'});
}
################################################################################
# DUMP ALL OBJECTS 2.x
################################################################################
sub dump_cfg_resource_2x {
my $icinga2_cfg = shift;
my $icinga2_global_macros = shift;
start_object_type_config_dump($icinga2_cfg, 'resource');
dump_config_line($icinga2_cfg, "set IcingaMacros = {");
foreach my $global_macro_2x_key (sort keys %{$icinga2_global_macros}) {
dump_config_line($icinga2_cfg, "\t\"$global_macro_2x_key\" = \"$icinga2_global_macros->{$global_macro_2x_key}\",");
}
dump_config_line($icinga2_cfg, "}");
end_object_type_config_dump($icinga2_cfg);
}
sub dump_cfg_obj_2x {
my $icinga2_cfg = shift;
my $cfg_obj_2x = shift;
dump_hosts_2x($icinga2_cfg, $cfg_obj_2x);
dump_services_2x($icinga2_cfg, $cfg_obj_2x);
dump_users_2x($icinga2_cfg, $cfg_obj_2x);
dump_notifications_2x($icinga2_cfg, $cfg_obj_2x);
dump_timeperiods_2x($icinga2_cfg, $cfg_obj_2x);
dump_groups_2x($icinga2_cfg, $cfg_obj_2x);
dump_commands_2x($icinga2_cfg, $cfg_obj_2x);
}
sub dump_hosts_2x {
my $icinga2_cfg = shift;
my $cfg_obj_2x = shift;
start_object_type_config_dump($icinga2_cfg, 'hosts');
#say Dumper($icinga2_cfg);
foreach my $host_2x_key (keys %{@$cfg_obj_2x{'host'}}) {
my $host_2x = @$cfg_obj_2x{'host'}->{$host_2x_key};
#say Dumper($host_2x);
#say "==============\n";
# function decides itsself if object or template
dump_host_2x($icinga2_cfg, $host_2x);
}
end_object_type_config_dump($icinga2_cfg);
}
sub dump_services_2x {
my $icinga2_cfg = shift;
my $cfg_obj_2x = shift;
start_object_type_config_dump($icinga2_cfg, 'services');
foreach my $service_2x_key (keys %{@$cfg_obj_2x{'service'}}) {
my $service_2x = @$cfg_obj_2x{'service'}->{$service_2x_key};
dump_service_2x($icinga2_cfg, $service_2x);
}
end_object_type_config_dump($icinga2_cfg);
}
sub dump_users_2x {
my $icinga2_cfg = shift;
my $cfg_obj_2x = shift;
start_object_type_config_dump($icinga2_cfg, 'users');
foreach my $user_2x_key (keys %{@$cfg_obj_2x{'user'}}) {
my $user_2x = @$cfg_obj_2x{'user'}->{$user_2x_key};
dump_user_2x($icinga2_cfg, $user_2x);
}
end_object_type_config_dump($icinga2_cfg);
}
sub dump_notifications_2x {
my $icinga2_cfg = shift;
my $cfg_obj_2x = shift;
start_object_type_config_dump($icinga2_cfg, 'notifications');
foreach my $notification_2x_key (keys %{@$cfg_obj_2x{'notification'}}) {
my $notification_2x = @$cfg_obj_2x{'notification'}->{$notification_2x_key};
dump_notification_2x($icinga2_cfg, $notification_2x);
}
end_object_type_config_dump($icinga2_cfg);
}
sub dump_timeperiods_2x {
my $icinga2_cfg = shift;
my $cfg_obj_2x = shift;
start_object_type_config_dump($icinga2_cfg, 'timeperiods');
foreach my $timeperiod_2x_key (keys %{@$cfg_obj_2x{'timeperiod'}}) {
my $timeperiod_2x = @$cfg_obj_2x{'timeperiod'}->{$timeperiod_2x_key};
dump_timeperiod_2x($icinga2_cfg, $timeperiod_2x);
}
end_object_type_config_dump($icinga2_cfg);
}
# XXX maybe split later
sub dump_groups_2x {
my $icinga2_cfg = shift;
my $cfg_obj_2x = shift;
start_object_type_config_dump($icinga2_cfg, 'groups');
if (!@$cfg_obj_2x{'hostgroup'}) {
goto SKIP_HOSTGROUPS;
}
foreach my $hostgroup_2x_key (keys %{@$cfg_obj_2x{'hostgroup'}}) {
my $hostgroup_2x = @$cfg_obj_2x{'hostgroup'}->{$hostgroup_2x_key};
dump_group_2x($icinga2_cfg, $hostgroup_2x);
}
SKIP_HOSTGROUPS:
if (!@$cfg_obj_2x{'servicegroup'}) {
goto SKIP_SERVICEGROUPS;
}
foreach my $servicegroup_2x_key (keys %{@$cfg_obj_2x{'servicegroup'}}) {
my $servicegroup_2x = @$cfg_obj_2x{'servicegroup'}->{$servicegroup_2x_key};
dump_group_2x($icinga2_cfg, $servicegroup_2x);
}
SKIP_SERVICEGROUPS:
if (!@$cfg_obj_2x{'usergroup'}) {
goto SKIP_USERGROUPS;
}
foreach my $usergroup_2x_key (keys %{@$cfg_obj_2x{'usergroup'}}) {
my $usergroup_2x = @$cfg_obj_2x{'usergroup'}->{$usergroup_2x_key};
dump_group_2x($icinga2_cfg, $usergroup_2x);
}
SKIP_USERGROUPS:
end_object_type_config_dump($icinga2_cfg);
}
sub dump_commands_2x {
my $icinga2_cfg = shift;
my $cfg_obj_2x = shift;
start_object_type_config_dump($icinga2_cfg, 'commands');
foreach my $command_2x_key (keys %{@$cfg_obj_2x{'command'}}) {
my $command_2x = @$cfg_obj_2x{'command'}->{$command_2x_key};
dump_command_2x($icinga2_cfg, $command_2x);
}
end_object_type_config_dump($icinga2_cfg);
}
################################################################################
# DUMP OBJECT 2.x
################################################################################
sub dump_service_2x {
my $icinga2_cfg = shift;
my $service_2x = shift;
my $object_type = "object"; # object or template
my $service_description = $service_2x->{__I2CONVERT_SERVICEDESCRIPTION};
# only dump templates, the objects will be directly created in host objects
if ($service_2x->{__I2CONVERT_IS_TEMPLATE} == 0) {
return;
}
#say Dumper($host_2x);
if ($service_2x->{__I2CONVERT_IS_TEMPLATE} == 1) {
$object_type = "template";
$service_description = $service_2x->{'__I2CONVERT_TEMPLATE_NAME'};
}
####################################################
# start, inherit from template?
####################################################
if (defined($service_2x->{'__I2CONVERT_USES_TEMPLATE'}) && $service_2x->{'__I2CONVERT_USES_TEMPLATE'} == 1) {
my $service_2x_templates = join '", "', @{$service_2x->{'__I2CONVERT_TEMPLATE_NAMES'}};
dump_config_line($icinga2_cfg, "$object_type Service \"$service_description\" inherits \"$service_2x_templates\" {");
} else {
dump_config_line($icinga2_cfg, "$object_type Service \"$service_description\" {");
}
####################################################
# display_name
####################################################
if(defined($service_2x->{'display_name'})) {
dump_config_line($icinga2_cfg, "\tdisplay_name = \"$service_2x->{'display_name'}\",");
}
####################################################
# macros
####################################################
if(defined($service_2x->{'__I2CONVERT_MACROS'}) && $service_2x->{'__I2CONVERT_MACROS'} != 0) {
dump_config_line($icinga2_cfg, "\tmacros = {");
foreach my $macro_key (keys %{$service_2x->{'__I2CONVERT_MACROS'}}) {
next if $macro_key =~ /USER/;
dump_config_line($icinga2_cfg, "\t\t$macro_key = \"$service_2x->{'__I2CONVERT_MACROS'}->{$macro_key}\",");
}
dump_config_line($icinga2_cfg, "\t},");
}
dump_config_line($icinga2_cfg, "");
####################################################
# check command
####################################################
if(defined($service_2x->{'__I2_CONVERT_CHECKCOMMAND_NAME'})) {
dump_config_line($icinga2_cfg, "\tcheck_command = \"$service_2x->{'__I2_CONVERT_CHECKCOMMAND_NAME'}\",");
}
####################################################
# event command
####################################################
if(defined($service_2x->{'__I2_CONVERT_EVENTCOMMAND_NAME'})) {
dump_config_line($icinga2_cfg, "\tevent_command = \"$service_2x->{'__I2_CONVERT_EVENTCOMMAND_NAME'}\",");
}
####################################################
# servicegroups
####################################################
if(defined($service_2x->{'servicegroups'}) && scalar(@{$service_2x->{'servicegroups'}}) > 0) {
#say Dumper($service_2x->{'servicegroups'});
my $servicegroups = join '", "', @{$service_2x->{'servicegroups'}};
if ($service_2x->{'__I2_CONVERT_SG_ADD'} == 1) {
dump_config_line($icinga2_cfg, "\tgroups += [ \"$servicegroups\" ],");
} else {
dump_config_line($icinga2_cfg, "\tgroups = [ \"$servicegroups\" ],");
}
#say Dumper($service_description);
#say Dumper($service_2x->{'servicegroups'});
}
####################################################
# servicedependencies (1.x deps)
####################################################
if(defined($service_2x->{'__I2CONVERT_PARENT_SERVICES'})) {
dump_config_line($icinga2_cfg, "\tservice_dependencies = [");
#say Dumper($service_2x);
# this is a hash with keys
foreach my $servicedep_key (keys %{$service_2x->{'__I2CONVERT_PARENT_SERVICES'}}) {
my $servicedep = $service_2x->{'__I2CONVERT_PARENT_SERVICES'}->{$servicedep_key};
dump_config_line($icinga2_cfg, "\t\t{ host = \"$servicedep->{'host'}\", service = \"$servicedep->{'service'}\" },");
}
dump_config_line($icinga2_cfg, "\t],");
}
####################################################
# notifications
####################################################
if(defined($service_2x->{'__I2CONVERT_NOTIFICATIONS'})) {
#say Dumper ($service_2x->{'__I2CONVERT_NOTIFICATIONS'});
# this is an array of notification objects
foreach my $service_notification_hash (@{$service_2x->{'__I2CONVERT_NOTIFICATIONS'}}) {
#say Dumper($service_notification_hash);
# this is a hash by unique key of the notification template, but all further attributes are seperatedly available too
foreach my $service_notification_key (keys %{$service_notification_hash}) {
my $service_notification = $service_notification_hash->{$service_notification_key};
#say Dumper($service_notification);
# skip everything not related to service notifications
next if ($service_notification->{'type'} ne 'service');
dump_config_line($icinga2_cfg, "\tnotifications[\"$service_notification->{'name'}\"] = {");
if (defined ($service_notification->{'templates'}) && @{$service_notification->{'templates'}} > 0) {
my $service_notification_templates = join '", "', @{$service_notification->{'templates'}};
dump_config_line($icinga2_cfg, "\t\ttemplates = [ \"$service_notification_templates\" ],");
}
if(defined($service_notification->{'users'}) && @{$service_notification->{'users'}} > 0) {
my $service_users = join '", "', @{$service_notification->{'users'}};
dump_config_line($icinga2_cfg, "\t\tusers = [ \"$service_users\" ],");
}
if(defined($service_notification->{'__I2CONVERT_NOTIFICATION_FILTERS'})) {
#say Dumper($service_notification);
foreach my $by (keys %{$service_notification->{'__I2CONVERT_NOTIFICATION_FILTERS'}}) {
next if !@{$service_notification->{'__I2CONVERT_NOTIFICATION_FILTERS'}->{$by}};
my $notification_filter;
if (grep /0/, @{$service_notification->{'__I2CONVERT_NOTIFICATION_FILTERS'}->{$by}}) {
$notification_filter = 0;
} else {
$notification_filter = "(". (join ' | ', @{$service_notification->{'__I2CONVERT_NOTIFICATION_FILTERS'}->{$by}}) .")";
}
dump_config_line($icinga2_cfg, "\t\tnotification_".$by."_filter = $notification_filter,");
}
}
if(defined($service_notification->{'__I2CONVERT_NOTIFICATION_INTERVAL'})) {
dump_config_line($icinga2_cfg, "\t\tnotification_interval = $service_notification->{'__I2CONVERT_NOTIFICATION_INTERVAL'},");
}
if(defined($service_notification->{'__I2CONVERT_NOTIFICATION_PERIOD'})) {
dump_config_line($icinga2_cfg, "\t\tnotification_period = \"$service_notification->{'__I2CONVERT_NOTIFICATION_PERIOD'}\",");
}
# this is set for escalations
if(defined($service_notification->{'__I2CONVERT_NOTIFICATION_TIMES'}) && $service_notification->{'__I2CONVERT_NOTIFICATION_TIMES'} != 0) {
dump_config_line($icinga2_cfg, "\t\ttimes = {");
dump_config_line($icinga2_cfg, "\t\t\tbegin = $service_notification->{'__I2CONVERT_NOTIFICATION_TIMES'}->{'begin'},");
dump_config_line($icinga2_cfg, "\t\t\tend = $service_notification->{'__I2CONVERT_NOTIFICATION_TIMES'}->{'end'}");
dump_config_line($icinga2_cfg, "\t\t},");
}
dump_config_line($icinga2_cfg, "\t},");
}
}
}
####################################################
# other service attributes, if set
####################################################
if(defined($service_2x->{'check_interval'})) {
dump_config_line($icinga2_cfg, "\tcheck_interval = $service_2x->{'check_interval'},");
}
if(defined($service_2x->{'retry_interval'})) {
dump_config_line($icinga2_cfg, "\tretry_interval = $service_2x->{'retry_interval'},");
}
if(defined($service_2x->{'max_check_attempts'})) {
dump_config_line($icinga2_cfg, "\tmax_check_attempts = $service_2x->{'max_check_attempts'},");
}
if(defined($service_2x->{'check_period'})) {
dump_config_line($icinga2_cfg, "\tcheck_period = \"$service_2x->{'check_period'}\",");
}
if(defined($service_2x->{'volatile'})) {
dump_config_line($icinga2_cfg, "\tvolatile = $service_2x->{'volatile'},");
}
if(defined($service_2x->{'flap_detection_enabled'})) {
dump_config_line($icinga2_cfg, "\tenable_flapping = $service_2x->{'flap_detection_enabled'},");
}
####################################################
# custom attr
####################################################
if(defined($service_2x->{'__I2CONVERT_CUSTOM_ATTR'}) && $service_2x->{'__I2CONVERT_CUSTOM_ATTR'} != 0) {
dump_config_line($icinga2_cfg, "\tcustom = {");
foreach my $custom_key (keys %{$service_2x->{'__I2CONVERT_CUSTOM_ATTR'}}) {
dump_config_line($icinga2_cfg, "\t\t$custom_key = \"$service_2x->{'__I2CONVERT_CUSTOM_ATTR'}->{$custom_key}\",");
}
dump_config_line($icinga2_cfg, "\t},");
dump_config_line($icinga2_cfg, "\tmacros += {");
foreach my $custom_key (keys %{$service_2x->{'__I2CONVERT_CUSTOM_ATTR'}}) {
if ($custom_key !~ /^_/) {
next;
}
my $key = substr($custom_key, 1);
dump_config_line($icinga2_cfg, "\t\t_SERVICE$key = \"$service_2x->{'__I2CONVERT_CUSTOM_ATTR'}->{$custom_key}\",");
}
dump_config_line($icinga2_cfg, "\t},");
}
dump_config_line($icinga2_cfg, "");
dump_config_line($icinga2_cfg, "}");
dump_config_line($icinga2_cfg, "\n");
}
sub dump_host_2x {
my $icinga2_cfg = shift;
my $host_2x = shift;
my $object_type = "object"; # object or template
my $host_name = $host_2x->{'host_name'}; # default, may be changed for templates
#say Dumper($host_2x);
if ($host_2x->{__I2CONVERT_IS_TEMPLATE} == 1) {
$object_type = "template";
$host_name = $host_2x->{'__I2CONVERT_TEMPLATE_NAME'};
}
####################################################
# start, inherit from template?
####################################################
if (defined($host_2x->{'__I2CONVERT_USES_TEMPLATE'}) && $host_2x->{'__I2CONVERT_USES_TEMPLATE'} == 1) {
my $host_2x_templates = join '", "', @{$host_2x->{'__I2CONVERT_TEMPLATE_NAMES'}};
dump_config_line($icinga2_cfg, "$object_type Host \"$host_name\" inherits \"$host_2x_templates\" {");
} else {
dump_config_line($icinga2_cfg, "$object_type Host \"$host_name\" {");
}
####################################################
# display_name
####################################################
if(defined($host_2x->{'display_name'})) {
dump_config_line($icinga2_cfg, "\tdisplay_name = \"$host_2x->{'display_name'}\",");
}
####################################################
# macros
####################################################
dump_config_line($icinga2_cfg, "");
if(defined($host_2x->{'__I2CONVERT_MACROS'}) && $host_2x->{'__I2CONVERT_MACROS'} != 0) {
dump_config_line($icinga2_cfg, "\tmacros = {");
foreach my $macro_key (keys %{$host_2x->{'__I2CONVERT_MACROS'}}) {
next if $macro_key =~ /USER/;
dump_config_line($icinga2_cfg, "\t\t$macro_key = \"$host_2x->{'__I2CONVERT_MACROS'}->{$macro_key}\",");
}
dump_config_line($icinga2_cfg, "\t},");
}
dump_config_line($icinga2_cfg, "");
####################################################
# hostgroups
####################################################
if(defined($host_2x->{'hostgroups'})) {
my $hostgroups = join '", "', @{$host_2x->{'hostgroups'}};
if ($host_2x->{'__I2_CONVERT_HG_ADD'} == 1) {
dump_config_line($icinga2_cfg, "\tgroups += [ \"$hostgroups\" ],");
} else {
dump_config_line($icinga2_cfg, "\tgroups = [ \"$hostgroups\" ],");
}
}
####################################################
# hostdependencies (1.x deps and parents combined)
####################################################
if(defined($host_2x->{'__I2CONVERT_PARENT_HOSTNAMES'})) {
my $hostdependency_hosts = join '", "', @{$host_2x->{'__I2CONVERT_PARENT_HOSTNAMES'}};
dump_config_line($icinga2_cfg, "\thost_dependencies = [ \"$hostdependency_hosts\" ],");
}
####################################################
# custom attr
####################################################
if(defined($host_2x->{'__I2CONVERT_CUSTOM_ATTR'}) && $host_2x->{'__I2CONVERT_CUSTOM_ATTR'} != 0) {
dump_config_line($icinga2_cfg, "\tcustom = {");
foreach my $custom_key (keys %{$host_2x->{'__I2CONVERT_CUSTOM_ATTR'}}) {
dump_config_line($icinga2_cfg, "\t\t$custom_key = \"$host_2x->{'__I2CONVERT_CUSTOM_ATTR'}->{$custom_key}\",");
}
dump_config_line($icinga2_cfg, "\t},");
dump_config_line($icinga2_cfg, "\tmacros += {");
foreach my $custom_key (keys %{$host_2x->{'__I2CONVERT_CUSTOM_ATTR'}}) {
if ($custom_key !~ /^_/) {
next;
}
my $key = substr($custom_key, 1);
dump_config_line($icinga2_cfg, "\t\t_HOST$key = \"$host_2x->{'__I2CONVERT_CUSTOM_ATTR'}->{$custom_key}\",");
}
dump_config_line($icinga2_cfg, "\t},");
}
####################################################
# hostcheck
####################################################
# this is magic, and must be set during conversion
if(defined($host_2x->{'__I2CONVERT_HOSTCHECK'})) {
dump_config_line($icinga2_cfg, "\tcheck = \"$host_2x->{'__I2CONVERT_HOSTCHECK'}\",");
}
####################################################
# host with no services - valid configuration
####################################################
if (!defined($host_2x->{'SERVICE'})) {
dump_config_line($icinga2_cfg, "}");
dump_config_line($icinga2_cfg, "\n");
return;
}
#say Dumper($host_2x->{'SERVICE'});
####################################################
# now all services with templates
####################################################
foreach my $service_2x_key (keys %{$host_2x->{'SERVICE'}}) {
my $service_2x = $host_2x->{'SERVICE'}->{$service_2x_key};
dump_config_line($icinga2_cfg, "\tservices[\"$service_2x->{__I2CONVERT_SERVICEDESCRIPTION}\"] = {");
####################################################
# templates
####################################################
if (defined($service_2x->{'__I2CONVERT_USES_TEMPLATE'}) && $service_2x->{'__I2CONVERT_USES_TEMPLATE'} == 1) {
my $service_2x_templates = join '", "', @{$service_2x->{'__I2CONVERT_TEMPLATE_NAMES'}};
dump_config_line($icinga2_cfg, "\t\ttemplates = [ \"$service_2x_templates\" ],")
}
####################################################
# display_name
####################################################
if(defined($service_2x->{'display_name'})) {
dump_config_line($icinga2_cfg, "\t\tdisplay_name = \"$service_2x->{'display_name'}\",");
}
dump_config_line($icinga2_cfg, "");
####################################################
# macros
####################################################
if(defined($service_2x->{'__I2CONVERT_MACROS'}) && $service_2x->{'__I2CONVERT_MACROS'} != 0) {
dump_config_line($icinga2_cfg, "\t\tmacros = {");
foreach my $macro_key (keys %{$service_2x->{'__I2CONVERT_MACROS'}}) {
next if $macro_key =~ /USER/;
dump_config_line($icinga2_cfg, "\t\t\t$macro_key = \"$service_2x->{'__I2CONVERT_MACROS'}->{$macro_key}\",");
}
dump_config_line($icinga2_cfg, "\t\t},");
}
####################################################
# check_command
####################################################
if(defined($service_2x->{'__I2_CONVERT_CHECKCOMMAND_NAME'})) {
dump_config_line($icinga2_cfg, "\t\tcheck_command = \"$service_2x->{'__I2_CONVERT_CHECKCOMMAND_NAME'}\",");
}
####################################################
# servicegroups
####################################################
if(defined($service_2x->{'servicegroups'}) && scalar(@{$service_2x->{'servicegroups'}}) > 0) {
#say Dumper($service_2x->{'servicegroups'});
my $servicegroups = join '", "', @{$service_2x->{'servicegroups'}};
if ($service_2x->{'__I2_CONVERT_SG_ADD'} == 1) {
dump_config_line($icinga2_cfg, "\t\tgroups += [ \"$servicegroups\" ],");
} else {
dump_config_line($icinga2_cfg, "\t\tgroups = [ \"$servicegroups\" ],");
}
}
####################################################
# notifications
####################################################
if(defined($service_2x->{'__I2CONVERT_NOTIFICATIONS'})) {
#say Dumper ($service_2x->{'__I2CONVERT_NOTIFICATIONS'});
# this is an array of notification objects
foreach my $service_notification_hash (@{$service_2x->{'__I2CONVERT_NOTIFICATIONS'}}) {
#say Dumper($service_notification_hash);
# this is a hash by unique key of the notification template, but all further attributes are seperatedly available too
foreach my $service_notification_key (keys %{$service_notification_hash}) {
my $service_notification = $service_notification_hash->{$service_notification_key};
#say Dumper($service_notification);
# skip everything not related to service notifications
next if ($service_notification->{'type'} ne 'service');
dump_config_line($icinga2_cfg, "\t\tnotifications[\"$service_notification->{'name'}\"] = {");
if (defined ($service_notification->{'templates'}) && @{$service_notification->{'templates'}} > 0) {
my $service_notification_templates = join '", "', @{$service_notification->{'templates'}};
dump_config_line($icinga2_cfg, "\t\t\ttemplates = [ \"$service_notification_templates\" ],");
}
if(defined($service_notification->{'users'}) && @{$service_notification->{'users'}} > 0) {
my $service_users = join '", "', @{$service_notification->{'users'}};
dump_config_line($icinga2_cfg, "\t\t\tusers = [ \"$service_users\" ],");
}
# this is set for escalations
if(defined($service_notification->{'__I2CONVERT_NOTIFICATION_TIMES'}) && $service_notification->{'__I2CONVERT_NOTIFICATION_TIMES'} != 0) {
dump_config_line($icinga2_cfg, "\t\t\ttimes = {");
dump_config_line($icinga2_cfg, "\t\t\t\tbegin = $service_notification->{'__I2CONVERT_NOTIFICATION_TIMES'}->{'begin'},");
dump_config_line($icinga2_cfg, "\t\t\t\tend = $service_notification->{'__I2CONVERT_NOTIFICATION_TIMES'}->{'end'}");
dump_config_line($icinga2_cfg, "\t\t\t},");
}
dump_config_line($icinga2_cfg, "\t\t},");
}
}
}
if(defined($host_2x->{'__I2CONVERT_NOTIFICATIONS'}) && $host_2x->{'__I2CONVERT_HOSTCHECK'} eq $service_2x->{__I2CONVERT_SERVICEDESCRIPTION}) {
#say Dumper ($host_2x->{'__I2CONVERT_NOTIFICATIONS'});
# this is an array of notification objects
foreach my $host_notification_hash (@{$host_2x->{'__I2CONVERT_NOTIFICATIONS'}}) {
#say Dumper($host_notification_hash);
# this is a hash by unique key of the notification template, but all further attributes are seperatedly available too
foreach my $host_notification_key (keys %{$host_notification_hash}) {
my $host_notification = $host_notification_hash->{$host_notification_key};
#say Dumper($host_notification);
# skip everything not related to host notifications
next if ($host_notification->{'type'} ne 'host');
dump_config_line($icinga2_cfg, "\t\tnotifications[\"$host_notification->{'name'}\"] = {");
if (defined ($host_notification->{'templates'}) && @{$host_notification->{'templates'}} > 0) {
my $host_notification_templates = join '", "', @{$host_notification->{'templates'}};
dump_config_line($icinga2_cfg, "\t\t\ttemplates = [ \"$host_notification_templates\" ],");
}
if(defined($host_notification->{'users'}) && @{$host_notification->{'users'}} > 0) {
my $host_users = join '", "', @{$host_notification->{'users'}};
dump_config_line($icinga2_cfg, "\t\t\tusers = [ \"$host_users\" ],");
}
dump_config_line($icinga2_cfg, "\t\t},");
}
}
}
####################################################
# other service attributes, if set
####################################################
if(defined($service_2x->{'check_interval'})) {
dump_config_line($icinga2_cfg, "\t\tcheck_interval = $service_2x->{'check_interval'},");
}
if(defined($service_2x->{'retry_interval'})) {
dump_config_line($icinga2_cfg, "\t\tretry_interval = $service_2x->{'retry_interval'},");
}
if(defined($service_2x->{'max_check_attempts'})) {
dump_config_line($icinga2_cfg, "\t\tmax_check_attempts = $service_2x->{'max_check_attempts'},");
}
if(defined($service_2x->{'check_period'})) {
dump_config_line($icinga2_cfg, "\tcheck_period = \"$service_2x->{'check_period'}\",");
}
if(defined($service_2x->{'flap_detection_enabled'})) {
dump_config_line($icinga2_cfg, "\t\tenable_flapping = $service_2x->{'flap_detection_enabled'},");
}
####################################################
# custom attr
####################################################
if(defined($service_2x->{'__I2CONVERT_CUSTOM_ATTR'}) && $service_2x->{'__I2CONVERT_CUSTOM_ATTR'} != 0) {
dump_config_line($icinga2_cfg, "\t\tcustom = {");
foreach my $custom_key (keys %{$service_2x->{'__I2CONVERT_CUSTOM_ATTR'}}) {
dump_config_line($icinga2_cfg, "\t\t\t$custom_key = \"$service_2x->{'__I2CONVERT_CUSTOM_ATTR'}->{$custom_key}\",");
}
dump_config_line($icinga2_cfg, "\t\t},");
dump_config_line($icinga2_cfg, "\t\tmacros += {");
foreach my $custom_key (keys %{$service_2x->{'__I2CONVERT_CUSTOM_ATTR'}}) {
if ($custom_key !~ /^_/) {
next;
}
my $key = substr($custom_key, 1);
dump_config_line($icinga2_cfg, "\t\t\t_SERVICE$key = \"$service_2x->{'__I2CONVERT_CUSTOM_ATTR'}->{$custom_key}\",");
}
dump_config_line($icinga2_cfg, "\t\t},");
}
dump_config_line($icinga2_cfg, "\t},");
dump_config_line($icinga2_cfg, "");
}
dump_config_line($icinga2_cfg, "}");
dump_config_line($icinga2_cfg, "\n");
}
sub dump_user_2x {
my $icinga2_cfg = shift;
my $user_2x = shift;
my $object_type = "object"; # object or template
my $user_name = $user_2x->{'contact_name'}; # default, may be changed for templates
#say Dumper($user_2x);
if ($user_2x->{__I2CONVERT_IS_TEMPLATE} == 1) {
$object_type = "template";
$user_name = $user_2x->{'__I2CONVERT_TEMPLATE_NAME'};
}
####################################################
# start, inherit from template?
####################################################
if (defined($user_2x->{'__I2CONVERT_USES_TEMPLATE'}) && $user_2x->{'__I2CONVERT_USES_TEMPLATE'} == 1) {
my $user_2x_templates = join '", "', @{$user_2x->{'__I2CONVERT_TEMPLATE_NAMES'}};
dump_config_line($icinga2_cfg, "$object_type User \"$user_name\" inherits \"$user_2x_templates\" {");
} else {
dump_config_line($icinga2_cfg, "$object_type User \"$user_name\" {");
}
if(defined($user_2x->{'display_name'})) {
dump_config_line($icinga2_cfg, "\tdisplay_name = \"$user_2x->{'display_name'}\",");
}
####################################################
# macros
####################################################
if(defined($user_2x->{'__I2CONVERT_MACROS'}) && $user_2x->{'__I2CONVERT_MACROS'} != 0) {
dump_config_line($icinga2_cfg, "\tmacros = {");
foreach my $macro_key (keys %{$user_2x->{'__I2CONVERT_MACROS'}}) {
next if $macro_key =~ /USER/;
dump_config_line($icinga2_cfg, "\t\t$macro_key = \"$user_2x->{'__I2CONVERT_MACROS'}->{$macro_key}\",");
}
dump_config_line($icinga2_cfg, "\t},");
}
dump_config_line($icinga2_cfg, "");
####################################################
# notifications
####################################################
if(defined($user_2x->{'notification_period'})) {
dump_config_line($icinga2_cfg, "\tnotification_period = \"$user_2x->{'notification_period'}\",");
}
if(defined($user_2x->{'__I2CONVERT_NOTIFICATION_FILTERS'})) {
foreach my $by (keys %{$user_2x->{'__I2CONVERT_NOTIFICATION_FILTERS'}}) {
next if !@{$user_2x->{'__I2CONVERT_NOTIFICATION_FILTERS'}->{$by}};
my $notification_filter;
if (grep /0/, @{$user_2x->{'__I2CONVERT_NOTIFICATION_FILTERS'}->{$by}}) {
$notification_filter = 0;
} else {
$notification_filter = "(". (join ' | ', @{$user_2x->{'__I2CONVERT_NOTIFICATION_FILTERS'}->{$by}}) .")";
}
dump_config_line($icinga2_cfg, "\tnotification_".$by."_filter = $notification_filter,");
}
}
if(defined($user_2x->{'notifications_enabled'})) {
dump_config_line($icinga2_cfg, "\tenable_notifications = \"$user_2x->{'notifications_enabled'}\",");
}
####################################################
# usergroups
####################################################
if(defined($user_2x->{'usergroups'})) {
#say Dumper($user_2x->{'usergroups'});
my $usergroups = join '", "', @{$user_2x->{'usergroups'}};
if ($user_2x->{'__I2_CONVERT_UG_ADD'} == 1) {
dump_config_line($icinga2_cfg, "\tgroups += [ \"$usergroups\" ],");
} else {
dump_config_line($icinga2_cfg, "\tgroups = [ \"$usergroups\" ],");
}
}
####################################################
# custom attr
####################################################
if(defined($user_2x->{'__I2CONVERT_CUSTOM_ATTR'}) && $user_2x->{'__I2CONVERT_CUSTOM_ATTR'} != 0) {
dump_config_line($icinga2_cfg, "\tcustom = {");
foreach my $custom_key (keys %{$user_2x->{'__I2CONVERT_CUSTOM_ATTR'}}) {
dump_config_line($icinga2_cfg, "\t\t$custom_key = \"$user_2x->{'__I2CONVERT_CUSTOM_ATTR'}->{$custom_key}\",");
}
dump_config_line($icinga2_cfg, "\t},");
dump_config_line($icinga2_cfg, "\tmacros += {");
foreach my $custom_key (keys %{$user_2x->{'__I2CONVERT_CUSTOM_ATTR'}}) {
if ($custom_key !~ /^_/) {
next;
}
my $key = substr($custom_key, 1);
dump_config_line($icinga2_cfg, "\t\t_CONTACT$key = \"$user_2x->{'__I2CONVERT_CUSTOM_ATTR'}->{$custom_key}\",");
}
dump_config_line($icinga2_cfg, "\t},");
}
dump_config_line($icinga2_cfg, "");
dump_config_line($icinga2_cfg, "}");
dump_config_line($icinga2_cfg, "\n");
}
sub dump_notification_2x {
my $icinga2_cfg = shift;
my $notification_2x = shift;
my $object_type = "object"; # object or template
my $notification_name = $notification_2x->{'__I2CONVERT_NOTIFICATION_NAME'}; # default, may be changed for templates
#say Dumper($notification_2x);
if ($notification_2x->{__I2CONVERT_IS_TEMPLATE} == 1) {
$object_type = "template";
$notification_name = $notification_2x->{'__I2CONVERT_NOTIFICATION_TEMPLATE_NAME'};
}
return if (!defined($notification_name));
####################################################
# start, inherit from template?
####################################################
if (defined($notification_2x->{'__I2CONVERT_USES_TEMPLATE'}) && $notification_2x->{'__I2CONVERT_USES_TEMPLATE'} == 1) {
my $notification_2x_templates = join '", "', @{$notification_2x->{'__I2CONVERT_TEMPLATE_NAMES'}};
dump_config_line($icinga2_cfg, "$object_type Notification \"$notification_name\" inherits \"$notification_2x_templates\" {");
} else {
dump_config_line($icinga2_cfg, "$object_type Notification \"$notification_name\" {");
}
if(defined($notification_2x->{'display_name'})) {
dump_config_line($icinga2_cfg, "\tdisplay_name = \"$notification_2x->{'display_name'}\",");
}
if(defined($notification_2x->{'__I2CONVERT_NOTIFICATION_COMMAND'})) {
#say Dumper($notifications_2x->{'notification_command'});
dump_config_line($icinga2_cfg, "\tnotification_command = \"$notification_2x->{'__I2CONVERT_NOTIFICATION_COMMAND'}\",");
}
if(defined($notification_2x->{'export_macros'})) {
#say Dumper($notification_2x->{'export_macros'});
my $export_macros = join '",\n"', @{$notification_2x->{'export_macros'}};
dump_config_line($icinga2_cfg, "\texport_macros = [ \"$export_macros\" ],");
}
if(defined($notification_2x->{'users'}) && @{$notification_2x->{'users'}} > 0) {
my $service_users = join '", "', @{$notification_2x->{'users'}};
dump_config_line($icinga2_cfg, "\tusers = [ \"$service_users\" ],");
}
if(defined($notification_2x->{'usergroups'}) && @{$notification_2x->{'usergroups'}} > 0) {
my $service_usergroups = join '", "', @{$notification_2x->{'usergroups'}};
dump_config_line($icinga2_cfg, "\tusergroups = [ \"$service_usergroups\" ],");
}
if(defined($notification_2x->{'__I2CONVERT_NOTIFICATION_FILTERS'})) {
foreach my $by (keys %{$notification_2x->{'__I2CONVERT_NOTIFICATION_FILTERS'}}) {
next if !@{$notification_2x->{'__I2CONVERT_NOTIFICATION_FILTERS'}->{$by}};
my $notification_filter;
if (grep /0/, @{$notification_2x->{'__I2CONVERT_NOTIFICATION_FILTERS'}->{$by}}) {
$notification_filter = 0;
} else {
$notification_filter = "(". (join ' | ', @{$notification_2x->{'__I2CONVERT_NOTIFICATION_FILTERS'}->{$by}}) .")";
}
dump_config_line($icinga2_cfg, "\tnotification_".$by."_filter = $notification_filter,");
}
}
if(defined($notification_2x->{'__I2CONVERT_NOTIFICATION_INTERVAL'})) {
dump_config_line($icinga2_cfg, "\tnotification_interval = $notification_2x->{'__I2CONVERT_NOTIFICATION_INTERVAL'},");
}
if(defined($notification_2x->{'__I2CONVERT_NOTIFICATION_PERIOD'})) {
dump_config_line($icinga2_cfg, "\tnotification_period = \"$notification_2x->{'__I2CONVERT_NOTIFICATION_PERIOD'}\",");
}
# this is set for escalations
if(defined($notification_2x->{'__I2CONVERT_NOTIFICATION_TIMES'}) && $notification_2x->{'__I2CONVERT_NOTIFICATION_TIMES'} != 0) {
dump_config_line($icinga2_cfg, "\ttimes = {");
dump_config_line($icinga2_cfg, "\t\tbegin = $notification_2x->{'__I2CONVERT_NOTIFICATION_TIMES'}->{'begin'},");
dump_config_line($icinga2_cfg, "\t\tend = $notification_2x->{'__I2CONVERT_NOTIFICATION_TIMES'}->{'end'}");
dump_config_line($icinga2_cfg, "\t},");
}
dump_config_line($icinga2_cfg, "");
dump_config_line($icinga2_cfg, "}");
dump_config_line($icinga2_cfg, "\n");
}
sub dump_timeperiod_2x {
my $icinga2_cfg = shift;
my $object_type = "object"; # object or template
my $timeperiod_2x = shift;
my $timeperiod_name = $timeperiod_2x->{'timeperiod_name'};
#say Dumper($timeperiod_2x);
if ($timeperiod_2x->{__I2CONVERT_IS_TEMPLATE} == 1) {
$object_type = "template";
$timeperiod_name = $timeperiod_2x->{'__I2CONVERT_TEMPLATE_NAME'};
}
####################################################
# start, inherit from template?
####################################################
if (defined($timeperiod_2x->{'__I2CONVERT_USES_TEMPLATE'}) && $timeperiod_2x->{'__I2CONVERT_USES_TEMPLATE'} == 1) {
my $timeperiod_2x_templates = join '", "', @{$timeperiod_2x->{'__I2CONVERT_TEMPLATE_NAMES'}};
dump_config_line($icinga2_cfg, "$object_type TimePeriod \"$timeperiod_name\" inherits \"$timeperiod_2x_templates\" {");
} else {
dump_config_line($icinga2_cfg, "$object_type TimePeriod \"$timeperiod_name\" {");
}
# display_name is seperated at first position
if(defined($timeperiod_2x->{'display_name'})) {
dump_config_line($icinga2_cfg, "\tdisplay_name = \"$timeperiod_2x->{'display_name'}\",");
}
dump_config_line($icinga2_cfg, "\tranges = {");
# dump all possible keys (there's no fixed string attr here)
foreach my $key (sort (keys %{$timeperiod_2x})) {
if ($key !~ /__I2CONVERT/ &&
$key ne 'alias' &&
$key ne 'name' &&
$key ne 'timeperiod_name' &&
$key ne 'display_name' &&
$key ne 'use'
) {
dump_config_line($icinga2_cfg, "\t\t\"$key\" \t= \"$timeperiod_2x->{$key}\",");
}
}
dump_config_line($icinga2_cfg, "\t},");
dump_config_line($icinga2_cfg, "");
dump_config_line($icinga2_cfg, "}");
dump_config_line($icinga2_cfg, "\n");
}
sub dump_group_2x {
my $icinga2_cfg = shift;
my $group_2x = shift;
my $group_name_attr = $group_2x->{__I2CONVERT_TYPE} . "_name";
my $group_name = $group_2x->{$group_name_attr};
my $group_type = ucfirst("$group_2x->{__I2CONVERT_TYPE}");
$group_type =~ s/group/Group/;
#say Dumper($group_2x);
dump_config_line($icinga2_cfg, "object $group_type \"$group_name\" {");
if(defined($group_2x->{'display_name'})) {
dump_config_line($icinga2_cfg, "\tdisplay_name = \"$group_2x->{'display_name'}\",");
}
####################################################
# custom attr
####################################################
if(defined($group_2x->{'__I2CONVERT_CUSTOM_ATTR'}) && $group_2x->{'__I2CONVERT_CUSTOM_ATTR'} != 0) {
dump_config_line($icinga2_cfg, "\tcustom = {");
foreach my $custom_key (keys %{$group_2x->{'__I2CONVERT_CUSTOM_ATTR'}}) {
dump_config_line($icinga2_cfg, "\t\t$custom_key = \"$group_2x->{'__I2CONVERT_CUSTOM_ATTR'}->{$custom_key}\",");
}
dump_config_line($icinga2_cfg, "\t},");
}
dump_config_line($icinga2_cfg, "");
dump_config_line($icinga2_cfg, "}");
dump_config_line($icinga2_cfg, "\n");
}
sub dump_command_2x {
my $icinga2_cfg = shift;
my $command_2x = shift;
my $command_name = $command_2x->{'__I2CONVERT_COMMAND_NAME'};
my $command_line = $command_2x->{'__I2CONVERT_COMMAND_LINE'};
my $command_type = ucfirst("$command_2x->{__I2CONVERT_COMMAND_TYPE}Command");
my $object_type = "object";
#say Dumper($command_2x);
# skip used commands
if ($command_2x->{'__I2_CONVERT_NOTIFICATION_COMMAND_USED'} == 1) {
say "Skipping already processed notification command" . Dumper($command_2x);
return;
}
if ($command_2x->{__I2CONVERT_IS_TEMPLATE} == 1) {
$object_type = "template";
$command_name = $command_2x->{'__I2CONVERT_TEMPLATE_NAME'};
}
####################################################
# start, inherit from template?
####################################################
if (defined($command_2x->{'__I2CONVERT_USES_TEMPLATE'}) && $command_2x->{'__I2CONVERT_USES_TEMPLATE'} == 1) {
my $command_2x_templates = join '", "', @{$command_2x->{'__I2CONVERT_TEMPLATE_NAMES'}};
dump_config_line($icinga2_cfg, "$object_type $command_type \"$command_name\" inherits \"$command_2x_templates\" {");
} else {
dump_config_line($icinga2_cfg, "$object_type $command_type \"$command_name\" {");
}
####################################################
# attributes
####################################################
if(defined($command_2x->{'display_name'})) {
dump_config_line($icinga2_cfg, "\tdisplay_name = \"$command_2x->{'display_name'}\",");
}
if(defined($command_line)) {
dump_config_line($icinga2_cfg, "\tcommand = \"$command_line\",");
}
dump_config_line($icinga2_cfg, "");
dump_config_line($icinga2_cfg, "}");
dump_config_line($icinga2_cfg, "\n");
}
=pod
UTILS
=cut
################################################################################
# HELPER FUNCTIONS
################################################################################
sub strip {
my $str = shift;
#strip trailing and leading whitespaces
$str =~ s/^\s+//;
$str =~ s/\s+$//;
return $str;
}
sub errlog {
my $err_lvl = shift;
my $log_str = shift;
if ($err_lvl > 0) {
print STDERR color("red"), "$log_str\n";;
} else {
print "$log_str\n";
}
}
sub escape_str {
my $str = shift;
$str =~s/\\/\\\\/g;
$str =~ s/"/\\"/g;
return $str;
}
sub escape_shell_meta {
my $str = shift;
$str =~ s/([;<>`'":&!#\$\[\]\{\}\(\)\*\|])/\\$1/g;
return $str;
}
sub debug {
my $dbg_str = shift;
our $dbg_lvl;
if ($dbg_lvl > 0) {
print "$dbg_str\n";
}
}
sub slurp {
my $file = shift;
if ( -f $file ) {
open ( my $fh, "<", $file ) or die "Could not open $file: $!";
return do {
<$fh>;
}
} elsif (! -r $file) {
print " ERROR: $file not readable. check permissions/user!\n"
} else {
print " ERROR: $file does not exist\n";
}
}
# stolen from http://stackoverflow.com/questions/7651/how-do-i-remove-duplicate-items-from-an-array-in-perl
sub uniq {
return keys %{{ map { $_ => 1 } @_ }};
}
sub str2arr_by_delim_without_excludes {
my $str = shift;
my $delim = shift;
my $sort = shift;
my $exclude = shift;
my @arr = ();
@arr = map { s/^\s+//; s/\s+$//; $_ }
grep { !/^!/ }
split (/$delim/, $str);
if ($sort == 1) {
@arr = sort (@arr);
}
return @arr;
}
sub strip_object_name {
my $obj_str = shift;
#$obj_str =~ s/[`~!\\\$%\^&\*|'"<>\?,\(\)=:]/_/g;
$obj_str =~ s/[:]/_/g;
return $obj_str;
}
sub print_sorted_hash {
my $hash = shift;
foreach my $key (sort keys %{$hash}) {
print "$key = $hash->{$key}\n";
}
}
Config Conversion: add conversion script Squashed Rebase, commit msg below. convertv1->v2 initial import refs #2743 itl: fix inclusion of notification.conf Config Conversion: command macros, templates refs #2743 WIP: try resolving the host->svc relation from templates and tricks not finished yet, there's heavy recursion required WIP configconversion: rewrite recursive lookup of linked host_name/service_description this is a mess. configs may just use "name" instead of "host_name" on type "host", same goes for service and service_descriptions. well, and if it's a template, "name" is populated too. the algorithm is not clean enough, and does not scale at all. there are bugs, and the hostgroup linking is not yet taken into account too. basics host -> service dumping of v2 config is implemented. refs #2743 ConfigConvert: split into modules, add objects.cache read, dump cmd line basically it works with - dumping a host with all services, using templates - check_command and all arguments/user macros translation - reading the objects.cache file location from icinga.cfg => this is reserved only for getting dedicated relations => we do not want to duplicate the work of xodtemplate.c in perl here - all important functions have been split up into their respective modules in order to support better re-usage (i.e. lconfexport) still missing important bits: - detect service->group<-host links and build templates - detect other ugly template methods - dump templates - the ugly deprecated normal|retry_check_interval mapping - timeperiod mapping - contact => user mapping - notifications rework (dedicated object) - dependencies, escalations (to be documented/implemented) - the infamous rest ConfigConvert: fix hash keys for convert/dump in perl 5.14 ConfigConvert: improved template handling, host/service dump 2x refactored code again, removed nested loop bottlenecks. still, the resolval of host_name/service_description requires more love. templates are now correctly detected, linked, and printed in 2x syntax. service templates are _not_ linked against host definitions in 2x as this is not possible. commands for hosts do not exist in 2x, therefore the command_name of an 1x host will be taken and guessed against its related services. if there's a match, the service will be linked as 2x 'hostcheck'. *_interval still needs proper handling, as the default interval is 1m, but 2x requires the identifier. ConfigConvert: refactor host_name/service_description link resolving previous algorithm was buggy, as well get_host_name always looked up host template objects, even if the template object was a service. this is now fixed by re-using the __TYPE attribute (set on Icinga 1x config read). the algorithm is now like |---------------------------------| \/ | svc_desc? | 0 / \ 1 | use tmpl? END -------> [svc_desc] | 0 / \ 1 | [undef] <- BROKEN getobjbyattr(['use'], __TYPE, svcdesc | | | getsvcdesc(obj) | | | -------------------------------------- refs #2743 ConvertConfig: add basic support for users, timeperiods, *groups as well as display_name as attribute to print something. contact => user mapping is added as well, manipulating the __TYPE var too. furthermore display_name on the host will be overwritten by the alias, if set. the groups don't dump much, only the display_name if set. their dump function is implemented generic, based on the __TYPE ConfigConvert: fix 4 little big thinkos on import/convert/dump 1) do not skip lines containing ;.* - there may actually be a comment on attr->val line. rather clean the line safely, and let the magic happen. 2) the type counter was implemented in order to have unique object ids by type, since 1x config does not provide us with such. problem - the function had a local var, saving the counter for each file, overwriting the hash attributes later on, starting again with 0. nasty bug, hard to figure out. fixed by feeding the type_cnt into the cfg_obj hive for now. revamped the state machine logic as well, like so I: define TYPE { --> __TYPE = TYPE II: attr val --> {type}->{type_cnt->cnt}->{attr} = val III: } --> type_cnt->cnt++ IV: <empty/comment> 3) also skip service templates (__IS_TEMPLATE) sooner, and do not try to calculate some host_name/service_description magic out of them - they just can't go down one level to the host, but only up with 'use'. 4) host and service templates on 2x config dump must check if __USES_TEMPLATE is actually set to 1, checking for defined is not enough. whatthecommit.com: A long time ago, in a galaxy far far away... refs #2743 ConfigConvert: add more host/service attributes, groups *_interval are automatically mapped, while the *groups attribute it converted into an array in order to easily stash more groups on it. still a todo - loop over existing groups and re-link their members into the respective objects, as 2.x only supports that location. fix for the host->service relation - use the already processed service 2x hashref in order to do some magic with host->{'SERVICE'} = service all 2x attributes are now encapsulated with "", to stay safe. todo - more generic config 2x dump. furthermore, add some more code comments for the respective sections. refs #2743 ConfigConvert: fix service linking not unique, add hostgroup->members re-link to host->hostgroups while working on the missing servicegroups and their facepalm members syntax (host1,svc1,host2,svc2 - awesome to parse), i figured that the whole detection of services by their attribute 'service_description' is not uniquely implemented. now the getter functions are duplicated, and not modular anymore, for the sake of having a service identified by its host_name + service_description. since that lookup is recursive, we must pass the host_name from the very beginning - otherwise we could accidently overwrite that with a template entry. the hostgroup re-linking works, also thanks to the fact that the inner object structure is just an array in the hashref, which allows smart push. contact/user groups are tricky again due to the different naming. further todo: sort all macros and hostservice links. right now this looks like chaos. refs #2743 ConfigConvert: basic servicegroup parsing method ConfigConvert: refactor multiple host_name's on service object detection this is ugly. a service object may contain 'host_name' with 'host1,host2,host3' and so on. treating this as string won't allow us to look into the template tree for valid host attributes, so we need to rewrite the getter function, allowing some special treatment on host_name arrays. for the service object preparation this means: while in the service loop, we need to loop again over all hosts, then dclone the hashref object for the service, and add the unique host_name attribute again. we might require some loop unrolling later, but for now this works as expected. refs #2743 ConvertConfig: disable debug output, some reformatting of dump ConfigConvert: refactor host-svc links, add full service/usergroup relinking due to the relinking afterwards we actually modify the service objects already prepared for 2x. since the *group magic depends on the host/services/users already prepared, we get a perpetuum mobile here with the host loop and resolving service relations. in order to safely fix the issue, there's a seperate host-service link loop after resolving all the other needed objects, which may cost a little bit more performance, but is safe and standalone, getting all previous modifications, migrations and also hacks. next to the refactored flow, the issue with resolving the correct servicegroups and stashing them into an array is solved, as well as converting the existing contactgroups to usergroups, as well as re-linking existing contactgroup->members to their new home, user->usergroups. note: when parsing the servicegroup members into an array, it must not be sorted afterwards, because we will shift 2 objects (host, service) in order to identify the correct service object for adding the servicegroup item. some code reorganisation too, plus a minor comma fix for the host address macro. the used 1x configuration will be addressed in 2743.cfg on icinga-core.git, with all future additions. refs #2743 ConfigConvert: *interval, max_check_attempts dumped as numbers see https://git.icinga.org/?p=icinga2.git;a=blob_plain;f=lib/icinga/icinga-type.conf;hb=HEAD refs #2743 ConfigConvert: fix undocumented contact templates conversion refs #2743 s/DumpIcinga2Cfg/ExportIcinga2Cfg/g just for Import/Export clarification. configconvert: use special prefix for internal attributes __I2CONVERT_ should do the trick, not to interfere with existing custom variables. configconvert: fix leftover from rename configconvert: properly escape strings with double quotes * command_line and command_args * notification commands * action_url, notes_url * notes * icon_image, icon_image_alt fixes #3931 configconvert: fix multiple templates usage, template object lookup error multiple host/service templates seperated by commas will cause some irritation, therefore this is treated as array internally, like group members. for the config export and "inherits", this is easy to fix. for the actual host_name lookup for service objects, this unveiled another bug when looping through all available templates - they are unique by 'name' and not by hostname/servicedesc, and require their very own getter - obj_get_tmpl_obj_by_tmpl_name() - which makes sure to return the object which is then looked into recursively if there's a host_name provided. possible bug for now - multiple service templates used and the host_name is defined in one of them. but that's not the usual case, and won't be supported for now. refs #2743 fixes #3932 configconvert: dump modular timperiod attr and values we need to ignore all other attributes which are to be cleaned sooner, just for safety. minor flaw: the keys are not sorted by weekdays now, but dumped like the hash provides them. fixes #4013 configconvert: support 'name' for groups' if *group_name is not set this is a fallback for broken configs only, and since i got some, added a small fix. refs #2743 configconvert: add service->hostgroup<-hostmembers conversion to template support WIP: there's multiple service->hostgroup relations - duplicated hosthg templates. must be fixed. refs #2743 refs #4016 configconvert: refactor service-hg-hosts for one hosthg template and multiple services we need to first loop over all services, getting the key ids, collecting them for each hostgroup_name (this is the unique identifier, because this is our hosthg template later on). then we loop over the collected unique hostgroups and build the 2.x template logic. the difference here is now, that the code is refactored, and loops over the service keys (getting their objects) only once, pushing their templates plus a relation to a single (1) hosthg-template. there's still some space for enhancements - the service template names as well as hosthg template names as well as service object names are generated with some hardcoded strings/numbers. refs #2743 refs #4016 configconversion: remove debug output configconvert: only dump macros if there are key-val pairs configconvert: code cleanup configconversion: explicitely check __I2CONVERT_USES_TEMPLATE == 1 when dumping service templates refs #4020 configconvert: refactor how services without host_name are handled this is mainly the cause when using hostgroup relations, or other object tricks. we still need the single loop run to convert all 2.x object attributes in one run, and deal with the missing attributes later on. configconvert: WIP notification conversion - refactored notification_commands parsing and storage - do not add duplicated notification_command names to the 2x notification object hive - create notifications based on the notification command linked from users - move host->service link magic to the end again refs #4008 configconvert: WIP notifications, how service/hosts relate to users and notifications there may be more than one notification for the user, and we already had the type in place. currently, that type is not taken into account (bug to fix). the rest with service -> user -> notification linking works in the first implementation. refs #4008 configconvert: final stage, rework notification type handling, fix naming amd template output the config dump must take care of which notification type is dumped, otherwise there would be yet another look required. Instead we are just passing a reference from users to hosts/services with all prepared notification attributes, and check them lazy on dump then. further fixe on the notification names - they are not unique by their command (now used on host/service scope, not user/contact) so we'll add some magic number as suffix. fixed template name dumping due to changed conversion vars too. refs #4008 configconvert: write config files, refactor how objects are exported structure is currently hardcoded, but may be changed to getopts later on. all files are overridden into conf/ (testing only right on). refactored export code, removed everything from Convert.pm, and export is now called from the script directly, wrapped all dump loops and added more function layers in betweed. open/close file handles take care of everything, next to a header on top of the files for getting to know their origin better (our hard work, ey?). refs #4036 configconvert: fix service description handling, it's now an inner conversion value previously, i've just re-used 'service_description' which isn't a valid 2.x attribute either. the lookup of service_description attribute in 1.x templates works already, but populated __I2CONVERT_SERVICEDESCRIPTION instead. in favor of having a generated value for all our objects, we'll use that one, also for dumping the configuration. even further, this is also changed for the service->hostgroup<-hosts conversion then. this also avoids writing empty service names into host->service objects. fixes #4019 configconvert: add small main config file for standalone tests refs #2743 configconvert: s/usergroups/groups/ for User config dump configconvert: add README configconvert: add ITL templates automatically (for services, notifications) this currently only makes sense for services and notifications. refs #4037 configconvert: fix timeperiod ranges, add template support for ITL inherits refs #4037 refs #4013 configconvert: update requirements, perl deps refs #2743 configconvert: fix import parser - timeperiods require regex (thx LConfImport) somehow, the Icinga2 import still fails with us holiday definitions. bug that's not yet determined whose bug that is. refs #4039 refs #4013 configconvert: add getopts and pod2usage incl help fixes #4041 configconvert: add notification_interval|period and refactor check|retry_interval vs normal|retry_check_interval refs #4008 configconvert: make host->service linked service a template object, referenced by the host this is not the best solution, as it it creates a service template for each host-service relation. if you have many host_name entries defined in your Icinga 1.x config for a service, this will create duplicates then. requires some more code refactoring, therefore leaving the issue open. refs #3933 configconvert: drop config file prefix 'icinga2' just makes it hard to work with. refs #4036 configconvert: code cleanup configconvert: export check_period for hosts/services refs #2743 configconvert: WIP dependencies - for host based on host_name and parents refs #4011 configconvert: s/Timeperiod/TimePeriod/g refs #4013 configconvert: add hostdependency with hostgroup_name hosts unrolling refs #4011 configconvert: add servicedependency support (host_name) missing - the loop fun with hostgroups. special config is here: https://git.icinga.org/?p=icinga-core.git;a=blob;f=tests/etc/2743.cfg;hb=refs/heads/next#l376 refs #4011 configconvert: add service dependency support for hostgroups well, a lot of looping required here: - loop through all child hostgroup hosts and get their hostnames - get the servive object based on each hostname (loop again) and the child service description - loop through all master hostgroup hosts and get their hostnames - get the master hostname based on each hostname (loop again) and the master service description, save that as service dependency given that implementation, dependencies may be converted for now. once we decide which options we support, we may add them as well. refs #4011 configconvert: add support for * as service host_name next to comma seperated list this is currently a proof of concept for services only, but should target all the object tricks later on. this will be done on host_name lookup and stored as array. http://docs.icinga.org/latest/en/objecttricks.html refs #4010 configconvert: refactor commaseperated list split ... to list of stripped strings into a function. fixes #4046 configconvert: handle excluded hosts, ignore them on split to array exclusions/exceptions cannot be treated by icinga2 the same way as in 1.x so it's better to not create unwanted objects. fixes #4007 configconvert: replace colon in object names with underscore it's a special delimiter and therefore forbidden. fixes #4225 configconvert: detect additive inheritance for *groups and use += on export i guess this can be used for host/service contact(groups) and other foo as well, so let's see. refs #4006 configconvert: add wildcard detection for 'members *' in hostgroup currently adds itself to the host's hostgroups array, but maybe a global template where all hosts inherit from, plus additive inheritance fit better here. refs #4010 configconvert: add a note about servicedeps wildcards todo refs #4010 configconvert: svg-hg-hst logic may contain multiple hgnames for services the object tricks section on the 1.x documentation contains everything which makes the code less readable even here. luckily the code is rather modular, so the extension for split string to array, and looping doesn't hurt much. refs #4010 configconvert: update README with usage and todos refs #2743 configconvert: fix perl 5.14, skip non-existing objects configconvert: escape display_name (may contain quotes) refs #3931 configconvert: refactor using command objects (check/notification) eventhandlers are missing. fixes #4305 configconvert: add volatile attribute fixes #4321 configconvert: eventhandler as EventCommand objects fixes #4327 update TODOs configconvert: remove obsolete itl notification template and check if defined when used. refs #2743 configconvert: write compat log too in icinga2 conf Revert "itl: fix inclusion of notification.conf" This reverts commit 02eb54dc1a818fdcafa6c85817e9f6341cf2831a. configconvert: fix thinko with check_command|event_command in service refs #2743 refs #4327 refs #4305
2013-03-12 17:40:53 +01:00
1;
__END__
# vi: sw=4 ts=4 expandtab :