Logstash with a constant queue

I started logstash using the following configurations:

Inside logstash.yml :

queue.type: persisted
queue.max_bytes: 8gb
queue.checkpoint.writes: 1

configuration file:

input {
    beats {
        port => "5043"
    }
}
filter {
    grok {
        match => {
            "message" => "%{COMBINEDAPACHELOG}"
        }
    }
    geoip {
        source => "clientip"
    }
}
output {
    elasticsearch {
        hosts => ["localhost:9200"]
        index => "test"
        document_type => "tw"
    }
}

I have such a situation.

  • Imagine that elastics search is disabled.

  • Now imagine that while elasticsearch is off, logstash received registration events

  • Now imagine we turn logstash off too

Now, if I enable logstash and elasticsearch, logstash does not send messages that were received during step 2 - when elasticsearch was disabled and logstash was receiving events .

+4
source share

Source: https://habr.com/ru/post/1669379/


All Articles