Syslog timestamp without a year?

Question

I fall asleep my magazines in Elasticsearch. Therefore, to create an index by the date of registration in it timestamp, I use the date filter as follows:

 date { "locale" => "en" match => ["timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss", "ISO8601"] target => "@timestamp" } 

I use logs from syslog and the timestamp syslog doest format has no year:

 # Syslog Dates: Month Day HH:MM:SS SYSLOGTIMESTAMP %{MONTH} +%{MONTHDAY} %{TIME} 

So, after using the date filter, the created index looks like logstash-2015.12.26 if I read the journal on December 26th, 2014. Since the timestamp is not available in the log, it selects the current year by default.

Any idea how to make the correct index?

Answer 1

Missing year in the line processed by Joda Time, Logstash is currently used by default during the year that the Logstash process was started. See github.com/logstash-plugins/logstash-filter-date error No. 3 . As a temporary workaround, add a temporary filter to add the correct year (2014) to the end of the timestamp field and edit your date filter including YYYY.

 filter { mutate { replace => ["timestamp", "%{timestamp} 2014"] } date { locale => "en" match => ["timestamp", "MMM d HH:mm:ss YYYY", "MMM dd HH:mm:ss YYYY", "ISO8601"] } } 

Answer 2

You can convert your date string to a date format using a date filter. By default, when you use the date filter, the date (or date-time) of your log will overwrite @timestamp. Thus, in your filter you do not need a target. You just use it if you want to convert a variable string to a date.

Example: match => ["timestamp", "MMM d HH: mm: ss", "MMM dd HH: mm: ss", "ISO8601"]

Answer 3

If the log files you upload have a year in the file name, you can extract it using the grok filter, create a new field with the date you pulled from syslog, plus the year with the file name.

An example of how to extract a date / time from a file name can be found here: Logstash: using date / time in a file name as an imported field