Elasticsearch - Derived Deep Metric

I have a web crawler that collects data and saves snapshots several times a day. My query has some clusters that group snapshots together per day and return the last snapshot of each day with top_hits.

The documents are as follows:

"_source": {
  "taken_at": "2016-02-01T11:27:09.184-03:00",
  ... ,
  "my_metric": 113
}

I would like to be able to calculate the derivative of some metric, say my_metric, the documents returned top_hits(i.e., the derivative of the last snapshots of each day my_metric).

Here is what I still have:

{
  "aggs": {
    "filtered_snapshots": {
      "filter": {
        // ...
      },
      "aggs" : {
        "grouped_data": {
          "date_histogram": {
            "field": "taken_at",
            "interval": "day",
            "format": "YYYY-MM-dd",
            "order": { "_key" : "asc" }
          },
          "aggs": {
            "resource_by_date": {
              "terms": { "field": "remote_id" },
              "aggs": {
                "latest_snapshots": {
                  "top_hits": { 
                    "sort": { "taken_at": { "order": "asc" }},
                    "size" : 1
                  }
                }
              }
            },
            "my_metric_deriv": {
              "derivative": {
                "buckets_path": "resource_by_date>latest_snapshots>my_metric" 
              }
            }
          }
        }
      }
    }
  }
}

I get the error "No aggregation [my_metric] for the path ..." with the request above.

bucket_path? bucket_path derivative , .

" ", , , . , .

+4

Source: https://habr.com/ru/post/1626900/


All Articles