How To Map An Elasticsearch Index Using The Python Client

Introduction

Python dictionaries can be used to create an Elasticsearch mapping schema, however, you must use Python Version 2 or 3 in order to be able to map Elasticsearch index with Python. You will also need to confirm the Elasticsearch cluster is up and running prior to beginning mapping an index with Elasticsearch. Make sure Kibana is installed and running on default port 5601 so you can verify the Python API requests made to the Elasticsearch cluster were successful when using Elasticsearch for mapping indexes with Python.

Prerequisites

  • Python Version 2 or 3 must be installed and working correctly on your machine. All of the examples in this tutorial use Python Version 3.x

  • Confirm the Elasticsearch cluster in running by checking the default port of 9200 by executing the following cURL request:

curl -XGET "http://localhost:9200"

Screenshot of a macOS terminal using a cURL request to GET the Elasticsearch cluster information

  • The Python low-level client for Elasticsearch must be installed via the PIP package manager for Python, as follows:
pip3 install Elasticsearch
  • Make sure Kibana is installed and running on default port is 5601, if you want to use the Kibana Console UI to verify the Python API requests made to the Elasticsearch cluster were successful.

  • Navigate to the port Kibana is running on in your browser and click on “Dev Tools”, represented by a “wrench” icon as shown below, and then click on “Console”:

Screenshot of the Kibana console UI running in the Chrome browser

How to Set Up the Python Script for the Elasticsearch Client

You can create a new Python script to execute API calls to Elasticsearch. You can use the touch command in a UNIX-based terminal, as follows:

sudo touch my_python_script.py

The file ends must end with .py for the server to recognizes it as a Python script.

Once inside the script, make certain you import the Elasticsearch client library. Next, create a new client instance of the library that you will use to create an index "_mapping" schema:

#!/usr/bin/env python3
#-*- coding: utf-8 -*-

from elasticsearch import Elasticsearch
elastic = Elasticsearch(hosts=["localhost"])
# or: elastic = Elasticsearch(hosts=["localhost"])

How to Create an Elasticsearch Mapping Schema Using a Python dictionary

To create a "_mapping " for the index you simply need to pass JSON data to the body parameter of the method. The Python dictionary should follow a blueprint similar to this:

mapping = {
"mappings": {
"properties": {
"some_string": {
"type": "text" # formerly "string"
},
"some_bool": {
"type": "boolean"
},
"some_int": {
"type": "integer"
},
"some_more_text": {
"type": "text"
}
}
}
}

NOTE: The "string" property field type has been depreciated and is now called the "text" type.

The "mappings" and "properties" fields are mandatory and all of the index’s property fields, or Python ‘keys’, must be nested inside of "properties" . The dictionary will then be passed to the Elasticsearch client’s indices class with a call to the create() method.

Changing the settings for a new Elasticsearch index:

You can also change the shards and replicas settings. Ensure the "settings" field is in its own separate field, or key, and that it is not nested in with the "mappings" field, as follows:

# mapping dictionary that contains the settings and
# _mapping schema for a new Elasticsearch index:
mapping = {
"settings": {
"number_of_shards": 2,
"number_of_replicas": 1
},
"mappings": {
"properties": {
"some_string": {
"type": "text" # formerly "string"
},
"some_bool": {
"type": "boolean"
},
"some_int": {
"type": "integer"
},
"some_more_text": {
"type": "text"
}
}
}
}

How to Pass the Python Dictionary to the Elasticsearch Client’s Create() Method

Here is what the basic layout of the indices.create() method’s API call looks like:

# create an index with the mapping passed to the 'body' parameter
indices.create(
index="__INDEX_NAME_GOES_HERE__",
body=MAPPING_DICT_GOES_HERE
)

There are only two required parameters when calling the method: 1) the index name passed as string into index; and 2) a Python dictionary passed into the body parameter. However, there is an option to ignore a parameter that uses integers representing HTTP status codes. Use the following command if you want to instruct Elasticsearch to ignore a particular HTTP error code:

# make an API call to the Elasticsearch cluster
# and have it return a response:
response = elastic_client.indices.create(
index="some_new_index",
body=mapping,
ignore=400 # ignore 400 already exists code
)

# print out the response:
print ('response:', response)

If executed properly, the Elasticsearch cluster should return a Python dictionary that has an "acknowledged" key with a value of True in the response, as follows:

response: {'acknowledged': True, 'shards_acknowledged': True, 'index': 'some_new_index'}

If you want to evaluate the command in a Python conditional statement, it would look like this:

if 'acknowledged' in response:
if response['acknowledged'] == True:
print ("INDEX MAPPING SUCCESS FOR INDEX:", response['index'])

# catch API error response
elif 'error' in response:
print ("ERROR:", response['error']['root_cause'])
print ("TYPE:", response['error']['type'])

How to Get the Mapping of the New Index Created with Python

Use the Kibana Console, or a cURL request in a terminal, as shown below to verify the index and its mapping were properly created:

# Kibana:
GET some_new_index/_mapping

# cURL request:
curl -XGET "localhost:9200/some_new_index/_mapping?pretty"

The GET request should return a JSON object in its response that looks like this:

{
"some_new_index" : {
"mappings" : {
"properties" : {
"some_bool" : {
"type" : "boolean"
},
"some_int" : {
"type" : "integer"
},
"some_more_text" : {
"type" : "text"
},
"some_string" : {
"type" : "text"
}
}
}
}
}

As Elasticsearch will return all of the "properties" in alphabetical order, the actual order of the "mappings" is irrelevant,

You can use Kibana or cURL to execute a GET request to Elasticsearch to obtain the _mapping of the new index:

Screenshot of Kibana and UNIX terminal making a GET request to an Elasticsearch cluster for _mapping of an index

Conclusion

In this tutorial you learned how to use Python scripts to create a mapping schema for Elasticsearch indexes. This function will only work with Python Version 2 or 3.

It is imperative to remember that there are only two required parameters when calling the method:

1) The index name passed as string into index. 2) A Python dictionary passed into the body parameter.

Also, bear in mind the "mappings" and "properties" fields are mandatory and all of the index’s property fields must be nested inside of "properties".

You can use the Kibana Console or a cURL request to verify the index and its mapping were properly created. As Elasticsearch will return all of the "properties" in alphabetical order, the actual order of the "mappings" is not important.

Pilot the ObjectRocket platform free for 30 Days

It's easy to get started. Imagine the time you'll save by not worrying about database management. Let's do this!

PILOT FREE FOR 30 DAYS

Keep in the know!

Subscribe to our emails and we’ll let you know what’s going on at ObjectRocket. We hate spam and make it easy to unsubscribe.