How to Prevent Indexing a Field in Elasticsearch Using Kibana

Introduction

There’s a good chance that not all of the data the typical user stores in Elasticsearch will end up being used in queries. For example, a user might be using Elasticsearch to store web session data. While it makes sense that a user would want to query on fields like the last update time and the session ID, there’s probably no need to query the session data itself.

It’s important to know that Elasticsearch indexes all data by default, whether it ultimately gets queried or not. To maintain better control over speed and efficiency, it’s generally recommended to disable indexing for any fields that will not be queried in searches. To avoid unnecessarily indexing fields, a few simple mapping changes can be made that will improve the efficiency of Elasticsearch. The following instructions provide a step-by-step guide to making these changes; to skip this tutorial and go straight to the code, simply click Just The Code.

Prerequisites

Before attempting to prevent indexing of a field in Elasticsearch, it’s important to confirm that the proper prerequisites are in place. For this task, only a couple of system requirements exist:

• Elasticsearch needs to be installed and running on the machine.
• In this tutorial, Kibana will be used to interact with Elasticsearch; to follow along with this step-by-step guide, it’s best to have Kibana installed and running as well. However, the instructions below can still be followed by users who know how to use cURL or other methods to interact with Elasticsearch.

Prevent Indexing a Field in Elasticsearch Using Kibana

Disable Indexing with Mapping “enabled”:false

To disable indexing on an Elasticsearch field, a change needs to be made in the mappings property. Elasticsearch uses mapping to define how a document and its fields are stored and indexed. In this case, the mapping parameter enabled will be set to false. The example below shows the creation of an index called demo_index which a demo type. Three additional fields are created: title, author, and age. In this example the indexing is disabled for the author field by using the "author": { "enabled": false }:

After making this mapping change, users will receive a confirmation that the action was successful in the terminal output, which should look like the following:

In this output, the acknowledged parameter tells the user if the demo_index index was successfully created. The shards_acknowledged parameter tells the user if the correct number of shard replicas were started for all shards in the index. The final parameter, index, simply confirms the name of the index that was created by the PUT command.

In addition to disabling an individual field from being indexed, an entire mapping type can be disabled at the object level. If this occurs, the document is stored and may be retrieved, but no part of its contents are indexed at all. One benefit of doing this is that no errors will be returned if data structures are changed within the object– this is none of the fields were ever indexed.

Conclusion

Although Elasticsearch indexes all fields when a document is stored, not all of those fields may need to be indexed. Only fields that are going to be used in queries should be indexed. The instructions detailed above provide a step-by-step guide to making the changes necessary to prevent indexing for a field in Elasticsearch. Applying this process to carefully-chosen fields in Elasticsearch can tighten up your database, making your searches faster and more efficient.

Just the Code

If you’re comfortable making the mapping changes without the step-by-step instructions and explanations included above, the following code contains everything needed to disable indexing of a field in Elasticsearch:

Prevent indexing of a field