Understanding elasticsearch query evaluation explains

Question

I am trying to decrypt the explanation API in elasticsearch answer. But a little lost. It's a little difficult for me to follow me. Any simple pointers or links that will explain JSON more specifically? I have an understanding of TF, IDF and cosine similarities in VSM. But we need some pointers to JSON more specifically. The ideal would be if I could find an explanation for this JSON as a simple mathematical expression.

{
  "_explanation": {
    "value": 7.937373,
    "description": "sum of:",
    "details": [
      {
        "value": 2.4789724,
        "description": "weight(FirstName:M80806 in 35) [PerFieldSimilarity], result of:",
        "details": [
          {
            "value": 2.4789724,
            "description": "score(doc=35,freq=1.0), product of:",
            "details": [
              {
                "value": 0.37350902,
                "description": "queryWeight, product of:",
                "details": [
                  {
                    "value": 6.6369815,
                    "description": "idf(docFreq=720, maxDocs=202323)"
                  },
                  {
                    "value": 0.056276944,
                    "description": "queryNorm"
                  }
                ]
              },
              {
                "value": 6.6369815,
                "description": "fieldWeight in 35, product of:",
                "details": [
                  {
                    "value": 1,
                    "description": "tf(freq=1.0), with freq of:",
                    "details": [
                      {
                        "value": 1,
                        "description": "termFreq=1.0"
                      }
                    ]
                  },
                  {
                    "value": 6.6369815,
                    "description": "idf(docFreq=720, maxDocs=202323)"
                  },
                  {
                    "value": 1,
                    "description": "fieldNorm(doc=35)"
                  }
                ]
              }
            ]
          }
        ]
      },
      {
        "value": 2.6825092,
        "description": "weight(FirstName:M8086 in 35) [PerFieldSimilarity], result of:",
        "details": [
          {
            "value": 2.6825092,
            "description": "score(doc=35,freq=1.0), product of:",
            "details": [
              {
                "value": 0.38854012,
                "description": "queryWeight, product of:",
                "details": [
                  {
                    "value": 6.9040728,
                    "description": "idf(docFreq=551, maxDocs=202323)"
                  },
                  {
                    "value": 0.056276944,
                    "description": "queryNorm"
                  }
                ]
              },
              {
                "value": 6.9040728,
                "description": "fieldWeight in 35, product of:",
                "details": [
                  {
                    "value": 1,
                    "description": "tf(freq=1.0), with freq of:",
                    "details": [
                      {
                        "value": 1,
                        "description": "termFreq=1.0"
                      }
                    ]
                  },
                  {
                    "value": 6.9040728,
                    "description": "idf(docFreq=551, maxDocs=202323)"
                  },
                  {
                    "value": 1,
                    "description": "fieldNorm(doc=35)"
                  }
                ]
              }
            ]
          }
        ]
      },
      {
        "value": 2.7758915,
        "description": "weight(FirstName:MHMT in 35) [PerFieldSimilarity], result of:",
        "details": [
          {
            "value": 2.7758915,
            "description": "score(doc=35,freq=1.0), product of:",
            "details": [
              {
                "value": 0.3952451,
                "description": "queryWeight, product of:",
                "details": [
                  {
                    "value": 7.0232153,
                    "description": "idf(docFreq=489, maxDocs=202323)"
                  },
                  {
                    "value": 0.056276944,
                    "description": "queryNorm"
                  }
                ]
              },
              {
                "value": 7.0232153,
                "description": "fieldWeight in 35, product of:",
                "details": [
                  {
                    "value": 1,
                    "description": "tf(freq=1.0), with freq of:",
                    "details": [
                      {
                        "value": 1,
                        "description": "termFreq=1.0"
                      }
                    ]
                  },
                  {
                    "value": 7.0232153,
                    "description": "idf(docFreq=489, maxDocs=202323)"
                  },
                  {
                    "value": 1,
                    "description": "fieldNorm(doc=35)"
                  }
                ]
              }
            ]
          }
        ]
      }
    ]
  }
}

Answer 1

Using the Ruby gem elasticsearch-explain-responsewill give you a more readable “explanation”, for example

require 'elasticsearch'
client = Elasticsearch::Client.new
result = client.explain index: "megacorp", type: "employee", id: "1", q: "last_name:Smith"
puts Elasticsearch::API::Response::ExplainResponse.new(result["explanation"]).render
#=>
1.0 = 1.0(fieldWeight)
  1.0 = 1.0(tf(1.0)) x 1.0(idf(2/3)) x 1.0(fieldNorm)
    1.0 = 1.0(termFreq=1.0)