Refactor environment for API connections

* Const renamed to `ApiEnvironment`
* Handling moved to ApiListener
* Now a property of ApiListener

doc/06-distributed-monitoring.md

@@ -116,8 +116,7 @@ you still need a [Host](09-object-types.md#objecttype-host) object.
 
 In case you are using the CLI commands later, you don't have to write
 this configuration from scratch in a text editor.
-The [ApiListener](09-object-types.md#objecttype-apilistener)
-object is used to load the SSL certificates and specify restrictions, e.g.
+The [ApiListener] object is used to load the SSL certificates and specify restrictions, e.g.
 for accepting configuration commands.
 
 It is also used for the [Icinga 2 REST API](12-icinga2-api.md#icinga2-api) which shares
@@ -2793,3 +2792,39 @@ Add the global zone `global-templates` in case it did not exist.
       global = true
     }
     EOF
+
+## Using Multiple Environments <a id="distributed-monitoring-environments"></a>
+
+In some cases it might be useful to run multiple Icinga instance on the same host. Two potential scenarios include:
+
+* running different versions of the same monitoring configuration (e.g. production and testing)
+* running disparate sets of checks for entirely unrelated monitoring environments (e.g. infrastructure and applications)
+
+Configuration is controlled via constants and attributes of the [ApiListener].
+
+Constant       | Attribute
+---------------|----------
+ApiEnvironment | environment
+ApiBindHost    | bind_host
+ApiBindPort    | bind_port
+
+In any case the constant is default value for the attribute, so that a direct configuration in the [ApiListener] object
+has more precedence. The constants have been created to allow the values to be set from the command line on startup.
+
+When Icinga establishes a TLS connection to another cluster instance it automatically uses the [SNI extension]
+to signal which endpoint it is attempting to connect to. On its own this can already be used to position multiple
+Icinga instances behind a load balancer.
+
+SNI example: `icinga2-client1.localdomain`
+
+However, if the environment is configured, Icinga will append the environment name to the SNI hostname like this:
+
+SNI example with environment: `icinga2-client1.localdomain:production`
+
+Middleware like loadbalancers or TLS proxies can read the SNI header and route the connection to the appropriate target.
+I.e., it uses a single externally-visible TCP port (usually 5665) and forwards connections to one or more Icinga
+instances which are bound to a local TCP port. It does so by inspecting the environment name that is sent as part of the
+SNI extension.
+
+[ApiListener]: 09-object-types.md#objecttype-apilistener
+[SNI Extension]: https://en.wikipedia.org/wiki/Server_Name_Indication

doc/09-object-types.md

@@ -67,6 +67,7 @@ Configuration Attributes:
   access\_control\_allow\_credentials   | Boolean               | **Deprecated.** Indicates whether or not the actual request can be made using credentials. Defaults to `true`. [(MDN docs)](https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS#Access-Control-Allow-Credentials)
   access\_control\_allow\_headers       | String                | **Deprecated.** Used in response to a preflight request to indicate which HTTP headers can be used when making the actual request. Defaults to `Authorization`. [(MDN docs)](https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS#Access-Control-Allow-Headers)
   access\_control\_allow\_methods       | String                | **Deprecated.** Used in response to a preflight request to indicate which HTTP methods can be used when making the actual request. Defaults to `GET, POST, PUT, DELETE`. [(MDN docs)](https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS#Access-Control-Allow-Methods)
+  environment                           | String                | **Optional.** Used as suffix in TLS SNI extension name; default from constant `ApiEnvironment`, which is empty.
 
 The attributes `access_control_allow_credentials`, `access_control_allow_headers` and `access_control_allow_methods`
 are controlled by Icinga 2 and are not changeable by config any more.

doc/17-language-reference.md

@@ -410,7 +410,7 @@ NodeName            |**Read-write.** Contains the cluster node name. Set to the
 RunAsUser           |**Read-write.** Defines the user the Icinga 2 daemon is running as. Set in the Icinga 2 sysconfig.
 RunAsGroup          |**Read-write.** Defines the group the Icinga 2 daemon is running as. Set in the Icinga 2 sysconfig.
 MaxConcurrentChecks |**Read-write.** The number of max checks run simultaneously. Defaults to `512`.
-Environment         |**Read-write.** The name of the Icinga environment. Included in the SNI host name when making outbound connections. Defaults to `production`.
+ApiEnvironment      |**Read-write**. The name of the Icinga environment for ApiListener. Included in the SNI host name when making outbound connections. Defaults to `production`.
 ApiBindHost         |**Read-write.** Overrides the default value for the ApiListener `bind_host` attribute. Not set by default.
 ApiBindPort         |**Read-write.** Overrides the default value for the ApiListener `bind_port` attribute. Not set by default.
 
@@ -454,7 +454,6 @@ SysconfDir          |**Read-only.** Contains the path of the sysconf directory.
 LocalStateDir       |**Read-only.** Contains the path of the local state directory. Defaults to `PrefixDir + "/var"`.
 RunDir              |**Read-only.** Contains the path of the run directory. Defaults to `LocalStateDir + "/run"`.
 
-
 Advanced runtime constants. Please only use them if advised by support or developers.
 
 Variable                   | Description

icinga-app/icinga.cpp

@@ -297,8 +297,6 @@ static int Main()
 	Application::DeclareConst("Concurrency", std::thread::hardware_concurrency());
 	Application::DeclareConst("MaxConcurrentChecks", Application::GetDefaultMaxConcurrentChecks());
 
-	ScriptGlobal::Set("Environment", "production");
-
 	ScriptGlobal::Set("AttachDebugger", false);
 
 	ScriptGlobal::Set("PlatformKernel", Utility::GetPlatformKernel());

lib/icinga/icingaapplication.cpp

@@ -81,8 +81,7 @@ void IcingaApplication::StatsFunc(const Dictionary::Ptr& status, const Array::Pt
 			{ "enable_perfdata", icingaapplication->GetEnablePerfdata() },
 			{ "pid", Utility::GetPid() },
 			{ "program_start", Application::GetStartTime() },
-			{ "version", Application::GetAppVersion() },
-			{ "environment", ScriptGlobal::Get("Environment", &Empty) }
+			{ "version", Application::GetAppVersion() }
 		}));
 	}
 

lib/remote/apilistener.cpp

@@ -388,16 +388,10 @@ void ApiListener::AddConnection(const Endpoint::Ptr& endpoint)
 
 	TcpSocket::Ptr client = new TcpSocket();
 
-	String serverName = endpoint->GetName();
-
-	String env = ScriptGlobal::Get("Environment", &Empty);
-	if (env != "" && env != "production")
-		serverName += ":" + env;
-
 	try {
 		endpoint->SetConnecting(true);
 		client->Connect(host, port);
-		NewClientHandler(client, serverName, RoleClient);
+		NewClientHandler(client, endpoint->GetName(), RoleClient);
 		endpoint->SetConnecting(false);
 	} catch (const std::exception& ex) {
 		endpoint->SetConnecting(false);
@@ -447,10 +441,16 @@ void ApiListener::NewClientHandlerInternal(const Socket::Ptr& client, const Stri
 
 	TlsStream::Ptr tlsStream;
 
+	String environmentName = GetEnvironment();
+	String serverName = hostname;
+
+	if (!environmentName.IsEmpty() && environmentName != "")
+		serverName += ":" + environmentName;
+
 	{
 		ObjectLock olock(this);
 		try {
-			tlsStream = new TlsStream(client, hostname, role, m_SSLContext);
+			tlsStream = new TlsStream(client, serverName, role, m_SSLContext);
 		} catch (const std::exception&) {
 			Log(LogCritical, "ApiListener")
 				<< "Cannot create TLS stream from client connection (" << conninfo << ")";

lib/remote/apilistener.ti

@@ -48,6 +48,10 @@ class ApiListener : ConfigObject
 		default {{{ return Application::GetConst("ApiBindPort", "5665"); }}}
 	};
 
+	[config] String environment {
+		default {{{ return Application::GetConst("ApiEnvironment"); }}}
+	};
+
 	[config] bool accept_config;
 	[config] bool accept_commands;