icingabeat/vendor/github.com/elastic/beats/filebeat/docs/filebeat-modules-options.asciidoc

[[configuration-filebeat-modules]]
== Specify which modules to run

NOTE: Using Filebeat modules is optional. You may decide to
<<configuration-filebeat-options,set up prospectors manually>> if you are using
a log file type that isn't supported, or you want to use a different setup.

Filebeat <<filebeat-modules,modules>> provide a quick way for you to get started
processing common log formats. They contain default configurations,
Elasticsearch ingest node pipeline definitions, and Kibana dashboards to help you
implement and deploy a log monitoring solution.

Filebeat provides a few different ways to enable modules. You can:

* <<enable-modules-d-configs>>
* <<enable-modules-cli>>
* <<enable-modules-config-file>>

include::../../libbeat/docs/shared-note-file-permissions.asciidoc[]

When you enable modules, you can also
<<specify-variable-settings,specify variable settings>> to change the default
behavior of the modules, and you can specify
<<advanced-settings,advanced settings>> to override prospector settings.

Before running Filebeat with modules enabled, make sure you also set up the
environment to use Kibana dashboards. See <<filebeat-modules-quickstart>> for
more information.

[float]
[[enable-modules-d-configs]]
=== Enable module configs in the `modules.d` directory

The `modules.d` directory contains default configurations for all the modules
available in Filebeat. You can enable or disable specific module configurations
under `modules.d` by running the
<<modules-command,`modules enable` or `modules disable`>> commands.

For example, to enable the `apache2` and `mysql` configs in the `modules.d`
directory, you use:

[source,shell]
----
./filebeat modules enable apache2 mysql
----

Then when you run Filebeat, it loads the corresponding module configurations
specified in the `modules.d` directory (for example, `modules.d/apache2.yml` and
`modules.d/mysql.yml`).

To see a list of enabled and disabled modules, run:

[source,shell]
----
./filebeat modules list
----

The default module configurations assume that the logs you’re harvesting are
in the location expected for your OS and that the behavior of the module is
appropriate for your environment. To change the default configurations, you need
to specify variable settings. See <<specify-variable-settings>>.

[float]
[[enable-modules-cli]]
=== Enable modules when you run Filebeat

To enable specific <<filebeat-modules,modules>> when you run Filebeat at the
command line, you can use the `--modules` flag. This approach works well when
you're getting started and want to specify different modules and settings each
time you run Filebeat. Any modules specified at the command line will be loaded
along with any modules that are enabled in the configuration file or `modules.d`
directory. If there's a conflict, the configuration specified at the command
line is used.

The following example shows how to enable and run the `nginx`,`mysql`, and
`system` modules.

[source,shell]
----
./filebeat -e --modules nginx,mysql,system
----

The default module configurations assume that the logs you’re harvesting are
in the location expected for your OS and that the behavior of the module is
appropriate for your environment. To change the default configurations, you need
to specify variable settings. See <<specify-variable-settings>>.

[float]
[[enable-modules-config-file]]
=== Enable module configs in the +{beatname_lc}.yml+ file

When possible, you should use the config files in the `modules.d` directory.

However, enabling <<filebeat-modules,modules>> directly in the config file is a
practical approach if you have upgraded from a previous version of {beatname_uc}
and don't want to move your module configs to the `modules.d` directory. You can
continue to configure modules in the +{beatname_lc}.yml+ file, but you won't be
able to use the `modules` command to enable and disable configurations because
the command requires the `modules.d` layout.

To enable specific modules in the +{beatname_lc}.yml+ config file, you can add
entries to the +{beatname_lc}.modules+ list. Each entry in the list begins with
a dash (-) and is followed by settings for that module.

The following example shows a configuration that runs the `nginx`,`mysql`, and
`system` modules.

[source,yaml]
----
filebeat.modules:
- module: nginx
- module: mysql
- module: system
----

The default module configurations assume that the logs you’re harvesting are
in the location expected for your OS and that the behavior of the module is
appropriate for your environment. To change the default configurations, you need
to specify variable settings. See <<specify-variable-settings>>.

[[specify-variable-settings]]
=== Specify variable settings

Each module and fileset has variables that you can set to change the default
behavior of the module, including the paths where the module looks for log
files. For example, the `var.paths` setting in the following example sets the
path for `nginx` access log files:

[source,yaml]
----
- module: nginx
  access:
    var.paths: ["/var/log/nginx/access.log*"]
----

To set the path for Nginx access log files at the command line, you use
the `-M` flag. For example:

[source,shell]
----
./filebeat -M "nginx.access.var.paths=[/var/log/nginx/access.log*]"
----

When you set variables at the command line, the variable name needs to include
the module and fileset name. You can specify multiple overrides. Each override
must start with `-M`.

Here you see how to use the `-M` flag along with the `--modules` flag. This
example shows how to set the paths to the access and error logs:

[source,shell]
----
./filebeat --modules nginx -M "nginx.access.var.paths=[/var/log/nginx/access.log*]" -M "nginx.error.var.paths=[/var/log/nginx/error.log*]"
----

For information about specific variables that you can set for each fileset,
see the <<filebeat-modules,documentation for the modules>>.

[[advanced-settings]]
=== Advanced settings

Behind the scenes, each module starts a Filebeat prospector. Advanced users
can add or override any prospector settings. For example, you can set
<<close-eof,close_eof>> to `true` in the module configuration:

[source,yaml]
----------------------------------------------------------------------
- module: nginx
  access:
    prospector:
      close_eof: true
----------------------------------------------------------------------

Or at the command line like this:

[source,shell]
----------------------------------------------------------------------
./filebeat -M "nginx.access.prospector.close_eof=true"
----------------------------------------------------------------------


Here you see how to use the `-M` flag along with the `--modules` flag:

[source,shell]
----------------------------------------------------------------------
./filebeat --modules nginx -M "nginx.access.prospector.close_eof=true"
----------------------------------------------------------------------


You can use wildcards to change variables or settings for multiple
modules/filesets at once. For example, the following command enables
`close_eof` for all the filesets in the `nginx` module:

[source,shell]
----------------------------------------------------------------------
./filebeat -M "nginx.*.prospector.close_eof=true"
----------------------------------------------------------------------

The following command enables `close_eof` for all prospectors created by any of
the modules:

[source,shell]
----------------------------------------------------------------------
./filebeat -M "*.*.prospector.close_eof=true"
----------------------------------------------------------------------