How to Use Percentile Aggregation in Elasticsearch with Curl

Introduction

An aggregation computation that comes up frequently when trying to figure out the distribution of your data is the percentile aggregation. Percentile aggregagtions are simple in Elasticsearch and this step-by-step example will show you an example of one that you can use as a basis for your own aggregation. In this step-by-step example we will interact with Elasticsearch using Curl commands to perform the percentile aggregation. If you’d just rather see the example code, click here to jump to Just the Code.

Note: The code will vary depending on all your system parameters but we hope to give you an idea of how this is done.

What is a Percentile Aggregation

Percentile aggregations are helpful to find outliers in data and figure out the distrubution of data. If you graduated in the top 10% of your class, that is a good example of percentile aggregation. Another way to look at it is to say that the 99th percentile is the value which is greater than 99% of all other values that appear in the dataset.

Prerequisites

Before we take a look at the how to perform a percentile aggregation, it’s important to mention a few prerequisites that need to be in place. For this task, the system requirements are minimal: Elasticsearch needs to be installed and running. Though it’s not required, it’s helpful to have some basic familiarity with the curl command. * In our example, we have Elasticsearch installed locally using the default port of 9200. If your Elasticsearch installation is running on a different server, you’ll need to modify your syntax accordingly.

Introduce our Example Data

Let’s look at an example that uses an index called store, which represents a small grocery store. This store index contains a type called products which lists the store’s products. To keep things simple, our example dataset will only contain a handful of products with just the following fields: id, price, quantity, and department. The code below shows the JSON used to create the dataset:

Here is the json we used to define the mapping if our index:

Use the Percentile Aggregation

Now let’s say we want to get a sense of our pricing at our store. We could use a percentile aggregation to get a sense of the distribution of our prices. The code to sum the quantities of each product is below. Let’s take a look and then we’ll dissect the code afterwards.

You can see what’s returned is an easy to read breakdown of our prices with our lowest price $0.99 at the low end (1st percentile) and $5.59 at the high end (99th percentile).

Now let’s go over the important parts of the code: We created an aggregator with "aggs" keyword. Then we gave it a name "price_by_percentile". This can be whatever name suits your application. Next we used "percentiles" to set the type of aggregator so Elasticsearch knows what calculation to perform with this aggregation. Lastly we specified which field we wanted to sum up ‘”price”`.

Note: In the curl url notice that we used size=0 because otherwise this query would return every product. With this parameter set, all we see is the aggregate information. We also specify the index and type in the url.

Conclusion

In this tutorial we demonstrated how to use Elasticsearch to use the percentile aggregation. There are many other operations you can perform with aggregations and you can consult the Elasticsearch documentation to learn more about it. The documentation is also useful if you need help with speciific syntax.

We hope you found tutorial helpful and you can apply it to your specific application. If you have questions or this didn’t work for you please reach out to us so we can help. Thank you.

Just the Code

If you’re already comfortable with curl and aggregations here’s all the code we used to demonstrate how to do a percentile aggregation with Elasticsearch and Curl.