How to Query Elasticsearch Documents In Python

Written by Data Pilot
May 09, 2019
  • Elasticsearch
Have a Database Problem? Speak with an Expert for Free
Get Started >>

Introduction

If you’re using Elasticsearch to store data, you may want to query your data from a Python script. Fortunately, the Python client for Elasticsearch makes it easy to communicate with Elasticsearch and query your indices. All it takes is a few lines of code to connect to Elasticsearch and query for the data you need. In this step-by-step tutorial, we’ll explain how to use the client to query for Elasticsearch documents in Python.

Prerequisites

Before we attempt to query for Elasticsearch documents in Python, it’s important to make sure certain prerequisites are in place. For this task, there are a few important system requirements:

  • You’ll need to have a decent understanding of Python and its syntax. It’s also helpful to know how JSON objects are structured.

  • You’ll need to have the low-level Python client for Elasticsearch installed. This client can run on either version 2 or 3 of Python, although our example in this article will use Python 3. If the client is not yet installed, just use Python’s pip3 to install the elasticsearch module:

1
pip3 install elasticsearch

Import the Python modules and connect to Elasticsearch

Once we’ve confirmed that all the system requirements are in place, we can start looking at the code. First, import the Elasticsearch client, Python’s JSON library, and the requests library, which is used to make queries to an Elasticsearch cluster:

1
2
3
4
5
#!/usr/bin/env python3
#-*- coding: utf-8 -*-

from elasticsearch import Elasticsearch
import json, requests

Next, create a new client instance of the Elasticsearch library:

1
2
# create a client instance of Elasticsearch
elastic_client = Elasticsearch()

Construct a Python dictionary to pass into the search() method

The Python client for Elasticsearch uses the same "query" parameters that one might use when making query requests in Kibana. The key difference is that in Python it will take the form of a dict class object:

1
2
3
4
5
6
7
8
# Python dictionary object representing an Elasticsearch JSON query:
search_param = {
    'query': {
        'match': {
            'field1': 'find me!'
        }
    }
}

A Python dictionary is simply an unordered collection of key: value pairs. The screenshot below shows an example of one.

Creating a Python dictionary in the IDLE interactive interpreter:

Screenshot of a dictionary class object in Python's IDLE environment

Pass a string to Python and convert it into a query dictionary

Python has a built-in JSON library that allows you to convert strings into Python dictionaries in JSON format. This is accomplished using the loads() method:

1
2
3
4
5
6
some_string = '{"hello" : "world"}'

# turn a JSON string into a dictionary:
some_dict = json.loads(some_string)
print (type(some_dict))
print (some_dict["hello"])

This makes it easy to pass AJAX string requests to Python, and then convert those Python strings into JSON objects for the Elasticsearch client.

Screenshot of Python's IDLE using the json.loads() method

Passing variables into a Python dictionary of an Elasticsearch query:

You can even pass a string into a query dictionary by putting brackets [] around the variable, as shown in the example below:

1
2
3
4
5
6
7
8
9
10
some_id = "Y4UYfmoB6VvZIesyYEBv"

# pass 'some_id' into the dictionary:
search_param = {
    "query": {
        "terms": {
            "_id": [ some_id ] # "for ID: Y4UYfmoB6VvZIesyYEBv"
        }
    }
}

Make an Elasticsearch query in Python by passing the dictionary to the search() method

Let’s take everything we just learned and create an Elasticsearch query to find two different documents using their respective IDs:

1
2
3
4
5
6
7
search_param = {
    "query": {
        "terms": {
            "_id": [ 1234, 42 ] # find Ids '1234' and '42'
        }
    }
}

When you pass your Python dictionary with the query, make sure to pass it in the body parameter. You can have the API call return a response object:

1
response = elastic_client.search(index="some_index", body=search_param)

Also, be sure that you specify the correct index when you make your query.

Conclusion

Querying for data is a common task when you’re using Elasticsearch as a search solution. Fortunately, it’s not difficult to query Elasticsearch from a Python script using the low-level Python client for Elasticsearch. Python’s JSON library and requests library also help make the task easy and keep your code clean and simple. With the step-by-step instructions included in this article, you’ll have no trouble querying Elasticsearch documents from a Python script.

In our article, we looked at the example code one section at a time. Here’s the complete Python script that can be used to query for Elasticsearch documents:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
#!/usr/bin/env python3
#-*- coding: utf-8 -*-

from elasticsearch import Elasticsearch
import json, requests

# create a client instance of Elasticsearch
elastic_client = Elasticsearch()

# create a Python dictionary for the search query:
search_param = {
    "query": {
        "terms": {
            "_id": [ 1234, 42 ] # find Ids '1234' and '42'
        }
    }
}

# get a response from the cluster
response = elastic_client.search(index="some_index", body=search_param)
print ('response:', response)


some_string = '{"field1" : "fine me!"}'

# turn a JSON string into a dictionary:
some_dict = json.loads(some_string)

# Python dictionary object representing an Elasticsearch JSON query:
search_param = {
    'query': {
        'match': {
            'field1': 'find me!'
        }
    }
}

# get another response from the cluster
response = elastic_client.search(index="some_index", body=search_param)
print ('response:', response)
Rate 1 Star2 Stars3 Stars4 Stars5 Stars

Pilot the ObjectRocket Platform Free!

Try Fully-Managed CockroachDB, Elasticsearch, MongoDB, PostgreSQL (Beta) or Redis.

Get Started
Related Topics:
Additional Information:

Keep in the know!

Subscribe to our emails and we’ll let you know what’s going on at ObjectRocket. We hate spam and make it easy to unsubscribe.