How to Query Elasticsearch Documents In Python
Introduction
If you’re using Elasticsearch to store data, you may want to query your data from a Python script. Fortunately, the Python client for Elasticsearch makes it easy to communicate with Elasticsearch and query your indices. All it takes is a few lines of code to connect to Elasticsearch and query for the data you need. In this step-by-step tutorial, we’ll explain how to use the client to query for Elasticsearch documents in Python.
Prerequisites
Before we attempt to query for Elasticsearch documents in Python, it’s important to make sure certain prerequisites are in place. For this task, there are a few important system requirements:
You’ll need to have a decent understanding of Python and its syntax. It’s also helpful to know how JSON objects are structured.
You’ll need to have the low-level Python client for Elasticsearch installed. This client can run on either version 2 or 3 of Python, although our example in this article will use Python 3. If the client is not yet installed, just use Python’s
pip3to install theelasticsearchmodule:
1 | pip3 install elasticsearch |
Import the Python modules and connect to Elasticsearch
Once we’ve confirmed that all the system requirements are in place, we can start looking at the code. First, import the Elasticsearch client, Python’s JSON library, and the requests library, which is used to make queries to an Elasticsearch cluster:
1 2 3 4 5 | #!/usr/bin/env python3 #-*- coding: utf-8 -*- from elasticsearch import Elasticsearch import json, requests |
Next, create a new client instance of the Elasticsearch library:
1 2 | # create a client instance of Elasticsearch elastic_client = Elasticsearch() |
Construct a Python dictionary to pass into the search() method
The Python client for Elasticsearch uses the same "query" parameters that one might use when making query requests in Kibana. The key difference is that in Python it will take the form of a dict class object:
1 2 3 4 5 6 7 8 | # Python dictionary object representing an Elasticsearch JSON query: search_param = { 'query': { 'match': { 'field1': 'find me!' } } } |
A Python dictionary is simply an unordered collection of key: value pairs. The screenshot below shows an example of one.
Creating a Python dictionary in the IDLE interactive interpreter:
Pass a string to Python and convert it into a query dictionary
Python has a built-in JSON library that allows you to convert strings into Python dictionaries in JSON format. This is accomplished using the loads() method:
1 2 3 4 5 6 | some_string = '{"hello" : "world"}' # turn a JSON string into a dictionary: some_dict = json.loads(some_string) print (type(some_dict)) print (some_dict["hello"]) |
This makes it easy to pass AJAX string requests to Python, and then convert those Python strings into JSON objects for the Elasticsearch client.
Passing variables into a Python dictionary of an Elasticsearch query:
You can even pass a string into a query dictionary by putting brackets [] around the variable, as shown in the example below:
1 2 3 4 5 6 7 8 9 10 | some_id = "Y4UYfmoB6VvZIesyYEBv" # pass 'some_id' into the dictionary: search_param = { "query": { "terms": { "_id": [ some_id ] # "for ID: Y4UYfmoB6VvZIesyYEBv" } } } |
Make an Elasticsearch query in Python by passing the dictionary to the search() method
Let’s take everything we just learned and create an Elasticsearch query to find two different documents using their respective IDs:
1 2 3 4 5 6 7 | search_param = { "query": { "terms": { "_id": [ 1234, 42 ] # find Ids '1234' and '42' } } } |
When you pass your Python dictionary with the query, make sure to pass it in the body parameter. You can have the API call return a response object:
1 | response = elastic_client.search(index="some_index", body=search_param) |
Also, be sure that you specify the correct index when you make your query.
Conclusion
Querying for data is a common task when you’re using Elasticsearch as a search solution. Fortunately, it’s not difficult to query Elasticsearch from a Python script using the low-level Python client for Elasticsearch. Python’s JSON library and requests library also help make the task easy and keep your code clean and simple. With the step-by-step instructions included in this article, you’ll have no trouble querying Elasticsearch documents from a Python script.
In our article, we looked at the example code one section at a time. Here’s the complete Python script that can be used to query for Elasticsearch documents:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 | #!/usr/bin/env python3 #-*- coding: utf-8 -*- from elasticsearch import Elasticsearch import json, requests # create a client instance of Elasticsearch elastic_client = Elasticsearch() # create a Python dictionary for the search query: search_param = { "query": { "terms": { "_id": [ 1234, 42 ] # find Ids '1234' and '42' } } } # get a response from the cluster response = elastic_client.search(index="some_index", body=search_param) print ('response:', response) some_string = '{"field1" : "fine me!"}' # turn a JSON string into a dictionary: some_dict = json.loads(some_string) # Python dictionary object representing an Elasticsearch JSON query: search_param = { 'query': { 'match': { 'field1': 'find me!' } } } # get another response from the cluster response = elastic_client.search(index="some_index", body=search_param) print ('response:', response) |
Pilot the ObjectRocket Platform Free!
Try Fully-Managed CockroachDB, Elasticsearch, MongoDB, PostgreSQL (Beta) or Redis.
Get Started