Rather than leaving stale connections about we tried to poll for data coming in
from InfluxDB and timeout if it didn't repond in a timely manner. This introduced
a race where the timeout triggers, a context switch occurs where data is actually
available and the TlsStream spins trying to asynchronously notify that data is
available, but which never gets read. Not only does this use up 100% of a core,
but it also slowly starves the system of handler threads at which point metrics
stop being delivered.
This basically removes the poll and timeout, any TLS socket erros should be
detected by TCP keep-alives.
Fixes#5460#5469
Exposes the TCP socket used to communicate with the InfluxDB server. When we are
expecing a response we can now call poll() on the socket to wait for data to become
available. If it doesn't in a user configurable timeout period we abort the request.
fixes#4927fixes#4941
Signed-off-by: Michael Friedrich <michael.friedrich@icinga.com>
Adds a new configuration variable in keeping with the graphite writer
which defaults to false to save network bandwidth. All metrics currently
supported by graphite are now available to InfluxDB. I added in some
formatting functions, to handle integers and booleans as we know and
control their types, and the supporting regexes in the sanity checker.
Updating to InfluxDB 0.13.X started giving 400 errors due to the missing
Host header in HTTP/1.1 requests. HttpRequest has been updated to auto-
magically add the host and port to these requests if not explicitly
stated by the client code.
The exception code has been cleaned up to break out of the function
early if such a condition is raised, this avoids unnecessarily executing
code which will ultimately fail.
fixes#11912
Signed-off-by: Gunnar Beutner <gunnar.beutner@netways.de>
Fixes a couple issues to do with line formatting of influx DB data points. All
keys and values need commas and white space escaping. Values are also checked
for type. If a numeric or scientific value is detected this is output as an
Influx floating point/scientific number. Booleans are detected and output in
a canonical format. All other values are strings, which have double quotes
escaped and the entire string is wrapped in double quotes. The handling of
thresholds has changed before this becomes officially released. These values
if available are passed to the accumulation function in a dictionary, said
dictionary builds a single data point with multiple fields, rather than the
existing 5 data points, thus saving bandwidth costs.
fixes#11904
Signed-off-by: Gunnar Beutner <gunnar.beutner@netways.de>
Adds an Icinga2 object to directly interface with InfluxDB's native HTTP API.
This supports optional basic authorization, and TLS transport. InfluxDB didn't
appear to like having the TLS stream kept open, so instead this object buffers
data points which are then flushed to InfluxDB as a batch write, either driven
by a configurable timeout or threshold.
As InfluxDB is a schema-less database the host and service templates are user
configurable allowing both the measurement field and tags to be set by the
end user via macro expansion. This allows access to tag fields from arbitrary
data associated with host.vars or service.vars. If a particular value is
unable to be resolved, the tag will be dropped and not transmitted to InfluxDB.
Also alters URL handling to omit array brackets when only a single value is
attached to a key, otherwise InfluxDB has a strop with non-standard syntax.
fixes#10480
Signed-off-by: Michael Friedrich <michael.friedrich@netways.de>
Simon Murray
Michael Friedrich
Gunnar Beutner