How can I get logstash to delete all events that do not match the regex group?

Question

I am trying to match event messages with multiple regular expressions. I was going to use the grep filter, but its deprecated, so I am trying to abandon the negation.

The functionality I'm looking for is to remove all events if the message does not match multiple regular expressions.

The filter below does not work, but it is checked separately, both expressions work fine. What am I missing?

filter { if ([message] !~ ' \[critical\]: ' or [message] !~ '\[crit\]: ') { drop { } } } 

Answer 1

You are using a regular expression in your conditional expression, but you are not passing the argument in the correct format. Doc shows this:

 if [status] =~ /^5\d\d/ { nagios { ... } } 

Note that the regex is not sorted and is surrounded by a slash.

Answer 2

I read a little more and continued to draw events with grok, adding a tag and discarding them at the end if the tag was not there:

 filter { grok { add_tag => [ "valid" ] match => [ "message", ".+ \[critical\]: ?(.+)", "message", ".+ \[crit\]: ?(.+) ", "message", '.+ (Deadlock found.+) ', "message", "(.+: Could not record email: .+) " ] } if "valid" not in [tags] { drop { } } mutate { remove_tag => [ "valid" ] } } 

Answer 3

 if "_grokparsefailure" in [tags] { drop {} }