[[configuration-heartbeat-options]]
== Set up monitors

++++
<titleabbrev>Set up Heartbeat monitors</titleabbrev>
++++

The `heartbeat.monitors` section of the +heartbeat.yml+ config file specifies
the list of `monitors` that Heartbeat uses to check your remote hosts to
determine if they are available. Each `monitor` item begins with a dash (-) and
specifies the type of monitor to use, the hosts to check, and other settings
that control Heartbeat behavior.

The following example configures three monitors, `icmp`, `tcp`, and `http`, and
demonstrates how to use TCP Echo and HTTP response verification:

[source,yaml]
----------------------------------------------------------------------
heartbeat.monitors:
- type: icmp
  schedule: '*/5 * * * * * *'
  hosts: ["myhost"]
- type: tcp
  schedule: '@every 5s'
  hosts: ["myhost:7"]  # default TCP Echo Protocol
  check.send: "Check"
  check.receive: "Check"
- type: http
  schedule: '@every 5s'
  urls: ["http://localhost:80/service/status"]
  check.response.status: 200
heartbeat.scheduler:
  limit: 10
----------------------------------------------------------------------

[float]
[[monitor-options]]
=== Monitor options

You can specify the following options in the `heartbeat.monitors` section of the
+heartbeat.yml+ config file. These options are the same for all monitors.
Each monitor type has additional configuration options that are specific to that
monitor type.

[float]
[[monitor-type]]
==== `type`

The type of monitor to run. One of:

* `icmp`: Uses an ICMP (v4 and v6) Echo Request to ping the configured hosts.
Requires root access. See <<monitor-icmp-options>>.
* `tcp`: Connects via TCP and optionally verifies the endpoint by sending and/or
receiving a custom payload. See <<monitor-tcp-options>>.
* `http`: Connects via HTTP and optionally verifies that the host returns the
expected response. See <<monitor-http-options>>.

The `tcp` and `http` monitor types both support SSL/TLS and some proxy
settings.


[float]
[[monitor-name]]
==== `name`

The monitor name. This value appears in the <<exported-fields,exported fields>>
under the `monitor` field as the job name and the `type` field as the job type.

[float]
[[monitor-enabled]]
==== `enabled`

A Boolean value that specifies whether the module is enabled. If the `enabled`
option is missing from the configuration block, the module is enabled by
default.

[float]
[[monitor-schedule]]
==== `schedule`

A cron-like expression that specifies the task schedule. For example:

* `*/5 * * * * * *` runs the task every 5 seconds (for example, at 10:00:00,
10:00:05, and so on).
* `@every 5s` runs the task every 5 seconds from the time when Heartbeat was
started.

The `schedule` option uses a cron-like syntax based on https://github.com/gorhill/cronexpr#implementation[this `cronexpr` implementation],
but adds the `@every` keyword.

[float]
[[monitor-ipv4]]
==== `ipv4`

A Boolean value that specifies whether to ping using the ipv4 protocol if
hostnames are configured. The default is `true`.

[float]
[[monitor-ipv6]]
==== `ipv6`

A Boolean value that specifies whether to ping using the ipv6 protocol
if hostnames are configured. The default is `true`.

[float]
[[monitor-mode]]
==== `mode`

If `mode` is `any`, the monitor pings only one IP address for a hostname. If
`mode` is `all`, the monitor pings all resolvable IPs for a hostname. The
`mode: all` setting is useful if you are using a DNS-load balancer and want to
ping every IP address for the specified hostname. The default is `any`.

[float]
[[monitor-timieout]]
==== `timeout`

The total running time for each ping test. This is the total time allowed for
testing the connection and exchanging data. The default is 16 seconds (16s).

If the timeout is exceeded, Heartbeat publishes a `service-down` event. If the
value specified for `timeout` is greater than `schedule`, intermediate checks
will not be executed by the scheduler.

[float]
[[monitor-fields]]
==== `fields`

Optional fields that you can specify to add additional information to the
output. For example, you might add fields that you can use for filtering log
data. Fields can be scalar values, arrays, dictionaries, or any nested
combination of these. By default, the fields that you specify here will be
grouped under a `fields` sub-dictionary in the output document. To store the
custom fields as top-level fields, set the `fields_under_root` option to true.
If a duplicate field is declared in the general configuration, then its value
will be overwritten by the value declared here.

[float]
[[monitor-fields-under-root]]
==== `fields_under_root`

If this option is set to true, the custom <<monitor-fields,fields>>
are stored as top-level fields in the output document instead of being grouped
under a `fields` sub-dictionary. If the custom field names conflict with other
field names added by Heartbeat, then the custom fields overwrite the other
fields.

[float]
[[monitor-tags]]
==== `tags`

A list of tags that will be sent with the monitor event. This setting is optional.

[float]
[[monitor-processors]]
==== `processors`

A list of processors to apply to the data generated by the monitor.

See <<filtering-and-enhancing-data>> for information about specifying
processors in your config.

[float]
[[monitor-watch-poll-file]]
==== `watch.poll_file`

experimental[]

The JSON file to watch for additional monitor configurations. The JSON file can
contain multiple objects, each of which specifies a different monitor config.
Heartbeat checks this file periodically and starts a new monitor instance for
each new JSON object added to the file. For example, imagine that you add
10 new entries to the JSON file, each for a different hostname. When Heartbeat
picks up the changes in the file, it merges the original config
(`hearbeat.yml`) plus the JSON objects, and starts a monitor for each new host
that you've configured. If you delete an object from the JSON file and it
doesn't exist in the main config, Heartbeat stops the monitor instance running
for that object.

Each monitor has a unique ID that's based on parameters like protocol, host,
and port. If two monitors have the same ID, Heartbeat uses the settings that
are defined in the last JSON object of the merged config. This means that
you can specify settings in the JSON file that overwrite the settings in
the main config. In this way, the configuration that you specify for the
monitor in the main Heartbeat config file acts like a default config that you
can live-reconfigure by specifying additional configurations in the external
JSON file.

Example configuration:

[source, yaml]
-------------------------------------------------------------------------------
heartbeat.monitors:
- type: tcp
  schedule: '*/5 * * * * * *'
  hosts: ["myhost"]
  watch.poll_file:
    path: {path.config}/monitors/dynamic.json
    interval: 5s
-------------------------------------------------------------------------------

*`path`*:: Specifies the path to the JSON file to check for updates.
*`interval`*:: Specifies how often Heartbeat checks the file for changes.

To reconfigure the settings specified in the example config, you could define
the following JSON objects in `dynamic.json`:

[source, json]
-------------------------------------------------------------------------------
{"hosts": ["myhost:1234"], "schedule": "*/15 * * * * * *"} <1>
{"hosts": ["tls://otherhost:479"], "ssl.certificate_authorities": ["path/to/ca/file.pem"]} <2>
-------------------------------------------------------------------------------
<1> Upon detecting the changes, Heartbeat stops the old monitor and then
restarts it with a schedule of 15 seconds between checks.
<2> Heartbeat starts a new monitor that uses a TLS-based connection with a
custom CA certificate.

[float]
[[monitor-icmp-options]]
=== ICMP options

These options configure Heartbeat to use ICMP (v4 and v6) Echo Requests to check
the configured hosts. These options are valid when the <<monitor-type,`type`>> is
`icmp`.

[float]
[[monitor-icmp-hosts]]
==== `hosts`

A list of hosts to ping.

[float]
[[monitor-icmp-wait]]
==== `wait`

The duration to wait before emitting another ICMP Echo Request. The default is 1
second (1s).

[float]
[[monitor-tcp-options]]
=== TCP options

These options configure Heartbeat to connect via TCP and optionally verify the
endpoint by sending and/or receiving a custom payload. These options are valid when
the <<monitor-type,`type`>> is `tcp`.

[float]
[[monitor-tcp-hosts]]
==== `hosts`

A list of hosts to ping. The entries in the list can be:

* A plain host name, such as `localhost`, or an IP address. If you specify this
option, you must also specify a value for <<monitor-tcp-ports,`ports`>>.  If the
monitor is configured to use SSL, Heartbeat establishes an SSL/TLS-based
connection. Otherwise, it establishes a plain TCP connection.
* A hostname and port, such as `localhost:12345`. Heartbeat connects
to the port on the specified host. If the monitor is
<<configuration-ssl,configured to use SSL>>, Heartbeat establishes an
SSL/TLS-based connection. Otherwise, it establishes a TCP connection.
* A full URL using the syntax `scheme://<host>:[port]`, where:
** `scheme` is one of `tcp`, `plain`, `ssl` or `tls`. If `tcp` or `plain` is
specified, Heartbeat establishes a TCP connection even if the monitor is
configured to use SSL. If `tls` or `ssl` is specified, Heartbeat establishes
an SSL connection. However, if the monitor is not configured to use SSL, the
system defaults are used (currently not supported on Windows).
** `host` is the hostname.
** `port` is the port number. If `port` is missing in the URL, the
<<monitor-tcp-ports,`ports`>> setting is required.

[float]
[[monitor-tcp-ports]]
==== `ports`

A list of ports to ping if the host specified in <<monitor-tcp-hosts,`hosts`>>
does not contain a port number.

Example configuration:

[source,yaml]
-------------------------------------------------------------------------------
- type: tcp
  schedule: '@every 5s'
  hosts: ["myhost"]
  ports: [80, 9200, 5044]
-------------------------------------------------------------------------------

[float]
[[monitor-tcp-check]]
==== `check`

An optional payload string to send to the remote host and the expected answer.
If no payload is specified, the endpoint is assumed to be available if the
connection attempt was successful. If `send` is specified without `receive`,
any response is accepted as OK. If `receive` is specified without `send`, no
payload is sent, but the client expects to receive a payload in the form of a
"hello message" or "banner" on connect.

Example configuration:

[source,yaml]
-------------------------------------------------------------------------------
- type: tcp
  schedule: '@every 5s'
  hosts: ["myhost"]
  ports: [7]
  check.send: 'Hello World'
  check.receive: 'Hello World'
-------------------------------------------------------------------------------


[float]
[[monitor-tcp-proxy-url]]
==== `proxy_url`

The URL of the SOCKS5 proxy to use when connecting to the server. The value
must be a URL with a scheme of socks5://.

If the SOCKS5 proxy server requires client authentication, then a username and
password can be embedded in the URL as shown in the example.

[source,yaml]
-------------------------------------------------------------------------------
  proxy_url: socks5://user:password@socks5-proxy:2233
-------------------------------------------------------------------------------

When using a proxy, hostnames are resolved on the proxy server instead of on
the client. You can change this behavior by setting the
`proxy_use_local_resolver` option.

[float]
[[monitor-tcp-proxy-use-local-resolver]]
==== `proxy_use_local_resolver`

A Boolean value that determines whether hostnames are resolved locally instead
of being resolved on the proxy server. The default value is false, which means
that name resolution occurs on the proxy server.

[float]
[[monitor-tcp-tls-ssl]]
==== `ssl`

The TLS/SSL connection settings.  If the monitor is
<<configuration-ssl,configured to use SSL>>, it will attempt an SSL
handshake. If `check` is not configured, the monitor will only check to see if
it can establish an SSL/TLS connection. This check can fail either at TCP level
or during certificate validation.

Example configuration:

[source,yaml]
-------------------------------------------------------------------------------
- type: tcp
  schedule: '@every 5s'
  hosts: ["myhost"]
  ports: [80, 9200, 5044]
  ssl:
    certificate_authorities: ['/etc/ca.crt']
    supported_protocols: ["TLSv1.0", "TLSv1.1", "TLSv1.2"]
-------------------------------------------------------------------------------


[float]
[[monitor-http-options]]
=== HTTP options

These options configure Heartbeat to connect via HTTP and optionally verify that
the host returns the expected response. These options are valid when the
<<monitor-type,`type`>> is `http`.

[float]
[[monitor-http-urls]]
==== `urls`

A list of URLs to ping.

Example configuration:

[source,yaml]
-------------------------------------------------------------------------------
- type: http
  schedule: '@every 5s'
  urls: ["http://myhost:80"]
-------------------------------------------------------------------------------


[float]
[[monitor-http-proxy-url]]
==== `proxy_url`

The HTTP proxy URL. This setting is optional. If not set, the HTTP_PROXY
environment variable is used.

[float]
[[monitor-http-username]]
==== `username`

The username for authenticating with the server. The credentials are passed
with the request. This setting is optional.

You need to specify credentials when your `check.response` settings require it.
For example, you can check for a 403 response (`check.response.status: 403`)
without setting credentials.

[float]
[[monitor-http-password]]
==== `password`

The password for authenticating with the server. This setting is optional.

[float]
[[monitor-http-tls-ssl]]
==== `ssl`

The TLS/SSL connection settings for use with the HTTPS endpoint. If you don't
specify settings, the system defaults are used.


Example configuration:

[source,yaml]
-------------------------------------------------------------------------------
- type: http
  schedule: '@every 5s'
  urls: ["https://myhost:80"]
  ssl:
    certificate_authorities: ['/etc/ca.crt']
    supported_protocols: ["TLSv1.0", "TLSv1.1", "TLSv1.2"]
-------------------------------------------------------------------------------


[float]
[[monitor-http-check]]
==== `check`

An optional `request` to send to the remote host and the expected `response`.

Example configuration:

[source,yaml]
-------------------------------------------------------------------------------
- type: http
  schedule: '@every 5s'
  urls: ["http://myhost:80"]
  check.request.method: HEAD
  check.response.status: 200
-------------------------------------------------------------------------------


Under `check.request`, specify these options:

*`method`*:: The HTTP method to use. Valid values are `"HEAD"`, `"GET"` and
`"POST"`.
*`headers`*:: A dictionary of additional HTTP headers to send.
*`body`*:: Optional request body content.

Under `check.response`, specify these options:

*`status`*:: The expected status code. If this setting is not configured or
it's set to 0, any status code other than 404 is accepted.
*`headers`*:: The required response headers.
*`body`*:: The required response body content.

The following configuration shows how to check the response when the body
contains JSON:

[source,yaml]
-------------------------------------------------------------------------------
- type: http
  schedule: '@every 5s'
  urls: ["https://myhost:80"]
check.request:
  method: GET
  headers:
    'X-API-Key': '12345-mykey-67890'
check.response:
  status: 200
  body: '{"status": "ok"}'
-------------------------------------------------------------------------------


[float]
[[monitors-scheduler]]
=== Scheduler options

You specify options under `heartbeat.scheduler` to control the behavior of the task
scheduler.

Example configuration:

[source,yaml]
-------------------------------------------------------------------------------
heartbeat.scheduler:
  limit: 10
  location: 'UTC-08:00'
-------------------------------------------------------------------------------

In the example, setting `limit` to 10 guarantees that only 10 concurrent
I/O tasks will be active. An I/O task can be the actual check or resolving an
address via DNS.

[float]
[[heartbeat-scheduler-limit]]
==== `limit`

The number of concurrent I/O tasks that Heartbeat is allowed to execute. If set
to 0, there is no limit. The default is 0.

Most operating systems set a file descriptor limit of 1024. For Heartbeat to
operate correctly and not accidentally block libbeat output, the value that you
specify for `limit` should be below the configured ulimit.


[float]
[[heartbeat-scheduler-location]]
==== `location`

The timezone for the scheduler. By default the scheduler uses localtime.



