Histogram of the date of maximum field amount through unique hosts

Question

I am trying to make a historical date from the sum of the maximum values ​​for a field through several values ​​for another field. Here is an example of two relevant documents:

         {
        "_index": "logstash-2014.02.06",
        "_type": "xyz",
        "_id": "HZ_2oaGvQvKWvsOLyYrGrw",
        "_score": 1,
        "_source": {
           "@version": "1",
           "@timestamp": "2014-02-05T16:01:01.260-08:00",
           "type": "xyz",
           "host": "compute-4.lab.solinea.com",
           "received_at": "2014-02-05 21:01:01 UTC",
           "received_from": "10.10.11.33",
           "total_widgets": 24,
        }
     },
     {
        "_index": "logstash-2014.02.06",
        "_type": "xyz",
        "_id": "HZ_2oaGvQvKWvsOLyYrGrx",
        "_score": 1,
        "_source": {
           "@version": "1",
           "@timestamp": "2014-02-05T16:01:01.260-08:00",
           "type": "xyz",
           "host": "compute-3.lab.solinea.com",
           "received_at": "2014-02-05 21:01:01 UTC",
           "received_from": "10.10.11.32",
           "total_widgets": 13,
        }
     }

In this case, I am looking for the sum (max (total_widgets)) for the unique hosts for this date bucket. I tried to set the date, but did not get what I was looking for. In this example:

{
   "query": {
      "range": {
         "@timestamp": {
            "gte": "2014-02-05T00:00:00+00:00",
            "lte": "2014-03-05T00:00:00+00:00"
         }
      }
   },
   "facets": {
      "total_widgets_facet": {
         "date_histogram": {
            "key_field": "@timestamp",
            "value_field": "total_widgets",
            "interval": "hour"
         },
         "facet_filter": {
            "term": {
               "type": "xyz"
            }
         }
      }
   }
}

I will return to the maximum value of 24, but I did not quite understand how to structure the query and facet so that I would look at the sum of the maximum number of "total_widgets" on all unique hosts for the time.

I definitely appreciate any suggestions ...

Answer 1

Elasticsearch 0.90.x, - 1.0.x :

{
   "query": {
      "bool": {
         "must": [
            {
               "range": {
                  "@timestamp": {
                     "from": "2014-02-07T00:00:00.000-00:00",
                     "to": "2014-02-07T23:59:59.999-00:00"
                  }
               }
            },
            {
               "term": {
                  "type": "xyz"
               }
            }
         ]
      }
   },
   "aggs": {
      "events_by_host": {
         "terms": {
            "field": "host.raw"
         },
         "aggs": {
            "events_by_date": {
               "date_histogram": {
                  "field": "@timestamp",
                  "interval": "hour"
               },
               "aggs": {
                  "max_total_widgets": {
                     "max": {
                        "field": "total_widgets"
                     }
                  },
                  "avg_total_widgets": {
                     "avg": {
                        "field": "total_widgets"
                     }
                  }
               }
            }
         }
      }
   }
}

: Elasticsearch Aggs Save the Day