How to Parse the Elasticsearch Query Data Returned by the Explain API In Python

Introduction

The Explain API in Elk stack provides detailed information about an Elasticsearch query’s score. The Explain API calculates a score “explanation” for the query of a particular document, providing useful feedback on whether or not a document matches a specific query. This tutorial will cover how to parse Elasticsearch query data with explain API in Python to make an API call in Python to the Elasticsearch cluster to retrieve data about a query and how to parse the returned information.

Prerequisites for Parsing Elasticsearch Query Data with Explain API in Python

  • Confirm the Elasticsearch cluster is running on the server by executing the following cURL request in a terminal window:
curl -XGET localhost:9200

The results should resemble the following:

Screenshot of a UNIX terminal making a cURL request to the Elasticsearch cluster * The Elasticsearch client must be installed on the same server making API requests to the cluster. Execute the following command to use the PIP3 package manager for Python 3 to install the low-level client:

pip3 install elasticsearch
  • A few documents on an Elasticsearch index for creating a query and for passing to the Elasticsearch.explain() method call. The following script represents the document used in this tutorial for the example query:
{
"_index" : "some_index",
"_type" : "_doc",
"_id" : "12345",
"_score" : 1.0,
"_source" : {
"string field" : "Object Rocket articles",
"integer field" : 42,
"boolean field" : false,
"timestamp" : "2019-08-10 19:57:39.195582"
}

The results should resemble the following:

Screenshot of an Elasticsearch document and index in Kibana

How to Import the Elasticsearch Client and Exceptions Libraries

Execute the following command to import the native JSON library. This will allow Python to format the JSON response returned by Elasticsearch to a more easily understood format:

# use the JSON library to prettify Elasticsearch JSON responses
import json

Now execute the following command to import the Elasticsearch low-level client as well as the client’s exceptions library that will catch any API errors returned by Elasticsearch:

# import the Elasticsearch client library
from elasticsearch import Elasticsearch, exceptions

How to Declare a Client Instance and an Elasticsearch Query Dictionary

This section will explain how to use the Elasticsearch library to create a client instance that can be used to make API calls to the cluster.

Execute the following command to have the Elasticsearch library return a client object for API calls:

# declare a client instance of the Python Elasticsearch library
client = Elasticsearch("http://localhost:9200")

How to declare a Python dictionary object for the Elasticsearch query

The Explain API requires a valid JSON query in the form of a Python dict object be passed to its method call. The following script will instantiate the query object outside of the method that will be passed later:

# declare a query dict object for the explain() method call
query = {"query": {"match" : {"string field" : "Object" }}}
print ("query:", query)

How to have the Explain() Method Return a Result ‘dict’ Object

The explain() method requires the client instance, the index named as a string and the _id and query body passed to it.

Execute the following command:

result = Elasticsearch.explain(client, index="some_index", id=12345, body=query)

How to use the json.dumps() method to indent and ameliorate the Elasticsearch API response

The JSON library’s dumps() method has an indent parameter that can be used to improve the JSON response returned by Elasticsearch. Execute the following command:

# print the JSON response from Elasticsearch with indentations
print (json.dumps(result, indent=4))

How to use the Elasticsearch Client’s ‘exceptions’ Library to Catch API Errors

The following is an example of how the Elasticsearch client’s exceptions library is used to catch any API errors returned that parses the error object’s info dictionary:

# if the query doesn't return a match it will return a 404 NotFoundError
try:
result = Elasticsearch.explain(client, index="some_index", id=12345, body=query)
except exceptions.NotFoundError as error:
# the elasticsearch.exceptions.NotFoundError object has a dict attribute
print (type(error))

How to parse the error object returned by Elasticsearch

The following command will catch the error object’s info attribute so that the result object will have dictionary values regardless of whether or not the API call was a success:

# get the dict attr to parse and evaluate
result = error.info
print ("explain() ERROR:", error)

How to Parse the Python Dictionary Returned by the API Call

As shown below, even if the results returned an API error, the result object will still have nested values of ["explanation"]["description"] that will explain why the API returned the specified results:

# access the "explanation" and "description" keys
print ("explanation description:", result["explanation"]["description"])
# prints --> "description": "weight(string field:object in 0) [PerFieldSimilarity], result of:"

How to confirm if the query response matched a document on the Elasticsearch index

The JSON response’s "matched" key will have a boolean string value of "true" or "false," depending on whether or not the query found a match. Execute the following command:

# print the "matched" key of the dict object
print ("query match:", result["matched"])

The results should resemble the following:

Screenshot of the Elasticsearch Explain API call in Python returning query data

Conclusion

This tutorial covered how to parse Elasticsearch query data with explain API in Python to make an API call in Python to the Elasticsearch cluster. The article specifically explained how to import the Elasticsearch client and exceptions libraries and how to have the explain() method return a result dict object. The tutorial also covered how to use the json.dumps() method to indent and ameliorate the Elasticsearch API response, how to use the Elasticsearch client’s exceptions library to catch API errors and how to parse the python dictionary returned by the API call. Remember that even if the results returned an API error when parsing the Python dictionary returned by the API call, then the result object will still have nested values of ["explanation"]["description"] to explain why the API returned those specific results.

Just the Code

#!/usr/bin/env python3
#-*- coding: utf-8 -*-

# use the JSON library to prettify Elasticsearch JSON responses
import json

# import the Elasticsearch client library
from elasticsearch import Elasticsearch, exceptions

# declare a client instance of the Python Elasticsearch library
client = Elasticsearch("http://localhost:9200")

# declare a query dict object for the explain() method call
query = {"query": {"match" : {"string field" : "Object" }}}
print ("query:", query)

# if the query doesn't return a match it will return a 404 NotFoundError
try:
result = Elasticsearch.explain(client, index="some_index", id=12345, body=query)
except exceptions.NotFoundError as error:
# the elasticsearch.exceptions.NotFoundError object has a dict attribute
print (type(error))

# get the dict attr to parse and evaluate
result = error.info
print ("explain() ERROR:", error)

# print the JSON response from Elasticsearch with indentations
print (json.dumps(result, indent=4))

# access the "explanation" and "description" keys
print ("explanation description:", result["explanation"]["description"])
# prints --> "description": "weight(string field:object in 0) [PerFieldSimilarity], result of:"

# print the "matched" key of the dict object
print ("query match:", result["matched"])

Pilot the ObjectRocket Platform Free!

Try Fully-Managed CockroachDB, Elasticsearch, MongoDB, PostgreSQL (Beta) or Redis.

Get Started

Keep in the know!

Subscribe to our emails and we’ll let you know what’s going on at ObjectRocket. We hate spam and make it easy to unsubscribe.