How to parse json in logstash / grok from a text string?

Question

I have a log file that looks like this (simplified)

Login example

MyLine data={"firstname":"bob","lastname":"the builder"} 

I would like to extract the json contained in the data and create two fields: one for the first, one for the last. However, I get the following:

 {"message":"Line data={\"firstname\":\"bob\",\"lastname\":\"the builder\"}\r","@version":"1","@timestamp":"2015-11-26T11:38:56.700Z","host":"xxx","path":"C:/logstashold/bin/input.txt","MyWord":"Line","parsedJson":{"firstname":"bob","lastname":"the builder"}} 

as you can see

 ..."parsedJson":{"firstname":"bob","lastname":"the builder"}} 

This is not what I need, I need to create fields for firstname and lastname in kibana, but logstash does not extract the fields using json filter.

LogStash Configuration

 input { file { path => "C:/logstashold/bin/input.txt" } } filter { grok { match => { "message" => "%{WORD:MyWord} data=%{GREEDYDATA:request}"} } json{ source => "request" target => "parsedJson" remove_field=>["request"] } } output { file{ path => "C:/logstashold/bin/output.txt" } } 

Any help is much appreciated, I'm sure I'm missing something simple

thanks

Answer 1

After your json filter, add another one called mutate to add two fields that you would take from the parsedJson field.

 filter { ... json { ... } mutate { add_field => { "firstname" => "%{[parsedJson][firstname]}" "lastname" => "%{[parsedJson][lastname]}" } } } 

For your example log line above this will produce:

 { "message" => "MyLine data={\"firstname\":\"bob\",\"lastname\":\"the builder\"}", "@version" => "1", "@timestamp" => "2015-11-26T11:54:52.556Z", "host" => "iMac.local", "MyWord" => "MyLine", "parsedJson" => { "firstname" => "bob", "lastname" => "the builder" }, "firstname" => "bob", "lastname" => "the builder" }