mirror of
https://github.com/Icinga/icingabeat.git
synced 2025-08-15 14:58:08 +02:00
69 lines
3.0 KiB
Plaintext
69 lines
3.0 KiB
Plaintext
//////////////////////////////////////////////////////////////////////////
|
|
//// This content is shared by all Elastic Beats. Make sure you keep the
|
|
//// descriptions here generic enough to work for all Beats that include
|
|
//// this file. When using cross references, make sure that the cross
|
|
//// references resolve correctly for any files that include this one.
|
|
//// Use the appropriate variables defined in the index.asciidoc file to
|
|
//// resolve Beat names: beatname_uc and beatname_lc.
|
|
//// Use the following include to pull this content into a doc file:
|
|
//// include::../../libbeat/docs/shared-config-ingest.asciidoc[]
|
|
//////////////////////////////////////////////////////////////////////////
|
|
|
|
[[configuring-ingest-node]]
|
|
== Parse logs by using ingest node
|
|
|
|
When you use Elasticsearch for output, you can configure {beatname_uc} to use
|
|
{elasticsearch}/ingest.html[ingest node] to pre-process documents
|
|
before the actual indexing takes place in Elasticsearch. Ingest node is a convenient processing option when you
|
|
want to do some extra processing on your data, but you do not require the full power of Logstash. For
|
|
example, you can create an ingest node pipeline in Elasticsearch that consists of one processor
|
|
that removes a field in a document followed by another processor that renames a field.
|
|
|
|
After defining the pipeline in Elasticsearch, you simply configure your Beat to use the pipeline. To configure
|
|
{beatname_uc}, you specify the pipeline ID in the `parameters` option under `elasticsearch` in the
|
|
+{beatname_lc}.yml+ file:
|
|
|
|
[source,yaml]
|
|
------------------------------------------------------------------------------
|
|
output.elasticsearch:
|
|
hosts: ["localhost:9200"]
|
|
pipeline: my_pipeline_id
|
|
------------------------------------------------------------------------------
|
|
|
|
For example, let's say that you've defined the following pipeline in a file named `pipeline.json`:
|
|
|
|
[source,json]
|
|
------------------------------------------------------------------------------
|
|
{
|
|
"description": "Test pipeline",
|
|
"processors": [
|
|
{
|
|
"lowercase": {
|
|
"field": "beat.name"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
------------------------------------------------------------------------------
|
|
|
|
To add the pipeline in Elasticsearch, you would run:
|
|
|
|
[source,shell]
|
|
------------------------------------------------------------------------------
|
|
curl -H 'Content-Type: application/json' -XPUT 'http://localhost:9200/_ingest/pipeline/test-pipeline' -d@pipeline.json
|
|
------------------------------------------------------------------------------
|
|
|
|
Then in the +{beatname_lc}.yml+ file, you would specify:
|
|
|
|
[source,yaml]
|
|
------------------------------------------------------------------------------
|
|
output.elasticsearch:
|
|
hosts: ["localhost:9200"]
|
|
pipeline: "test-pipeline"
|
|
------------------------------------------------------------------------------
|
|
|
|
When you run {beatname_uc}, the value of `beat.name` is converted to lowercase before indexing.
|
|
|
|
For more information about defining a pre-processing pipeline, see the {elasticsearch}/ingest.html[Ingest Node]
|
|
documentation.
|