Logstash filter, how to create two (or higher) outputs for one input

Question

I get one json through http poller

{ "id":12345 "name":"", "lastname":"", "age":12, "address":{"city":"XXXX" , "street":"ZZZZ" } } 

and I would like this to create two documents in my release:

man:

  { "id":12345 "name":"", "lastname":"", "age":12 } 

address:

  { "city":"XXXX" , "street":"ZZZZ" } 

As a result, I got one input event

on the input phase receiving one input:

 input { http_poller { urls => { test1 => "http://localhost:8080" } } 

in the filter phase, I would like to:

• create a human event (mark it as P)
• create an address event (mark it as A)

in the output phase, I would like to:

• send P to type P in ES
• send type A to type ES

Answer 1

You can achieve this with clone filter .

First, you need to install the plugin, which is not supplied by default:

 bin/logstash-plugin install logstash-filter-clone 

You can then reconfigure Logstash as follows:

 input { http_poller { urls => { test1 => "http://localhost:8080" } type => "A" } } filter { clone { clones => [ "P" ] add_tag => [ "P" ] } if [type] == "P" { mutate { remove_field => [ "address" ] } } else { mutate { add_tag => [ "A" ] remove_field => [ "id", "name", "lastname", "age" ] } } } output { elasticsearch { hosts => ["localhost:9200"] document_type => "%{type}" } }