How to Use the Extended Stats Aggregation in Elasticsearch using NodeJS
Introduction
Elasticsearch aggregations allow you to compute averages, sums, maximums, minimums, standard deviations and more. You can get all these individually but the extended stats aggregation lets you get a set of the most common aggregations you could want in one call. If you’re looking to get a baseline feel for some of your data this aggregation could be extremely helpful. We’ll show you exactly what aggregations it returns and a step-by-step tutorial of how in Javascript running on top of NodeJS. If you’d just rather see the example code, click here to jump to Just the Code.
Prerequisites
Before we show you how to get the extended stats with Elasticsearch in Javascript, it’s important to make sure a few prerequisites are in place. There are only a few of system requirements for this task:
NodeJS needs to be installed
The elasticsearch npm module installed.
A simple npm install elasticsearch
should work in most cases.
Elasticsearch also needs to be installed and running.
* In our example, we have Elasticsearch installed locally using the default port of 9200. If your Elasticsearch installation is running on a different server, you’ll need to modify your javascript syntax accordingly.
What aggregations does extended stats provide?
Here is a list of the aggregations that the extended stats returns: count mimimum maximum average sum sum of squares variance standard deviation * standard deviation bounds
Our Demo Index
We love to show by example. In our example for this extended stats aggregation we will use an index representing a small grocery store called store
. The store
index contains the type products
which lists all the stores products. To keep it simple our dataset only has a small number of products with just a few fields including the id, price, quantity, and department. Here is the json we used to create our dataset:
id | name | price | quantity | department |
---|---|---|---|---|
1 | Multi-Grain Cereal | 4.99 | 4 | Packaged Foods |
2 | 1lb Ground Beef | 3.99 | 29 | Meat and Seafood |
3 | Dozen Apples | 2.49 | 12 | Produce |
4 | Chocolate Bar | 1.29 | 2 | Packaged Foods, Checkout |
5 | 1 Gallon Milk | 3.29 | 16 | Dairy |
6 | 0.5lb Jumbo Shrimp | 5.29 | 12 | Meat and Seafood |
7 | Wheat Bread | 1.29 | 5 | Bakery |
8 | Pepperoni Pizza | 2.99 | 5 | Frozen |
9 | 12 Pack Cola | 5.29 | 6 | Packaged Foods |
10 | Lime Juice | 0.99 | 20 | Produce |
11 | 12 Pack Cherry Cola | 5.59 | 5 | Packaged Foods |
12 | 1 Gallon Soy Milk | 3.39 | 10 | Dairy |
13 | 1 Gallon Vanilla Soy Milk | 3.49 | 9 | Dairy |
14 | 1 Gallon Orange Juice | 3.29 | 4 | Juice |
Here is the json we used to define the mapping:
1 2 3 4 5 6 7 8 9 10 11 12 | { "mappings": { "products": { "properties" : { "name": { "type": "text"}, "price": { "type": "double"}, "quantity": { "type": "integer"}, "department": { "type": "keyword"} } } } } |
An Example of using the Extended Stats Aggregation
So an example for our store
index might be to use it on the price
field in our extended stats aggregation to quickly and easily get the average price as well as the maximum and minimum. Here’s the code to do it:
File index.js
:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | var elasticsearch = require("elasticsearch"); var client = new elasticsearch.Client({ hosts: ["http://localhost:9200"] }); /* Get Extended Stats Aggregation of price field */ client.search({ size: 0, index: 'store', type: 'products', body: { "aggs" : { "price_stats" : { "extended_stats" : { "field" : "price" } } } } }).then(function(resp) { console.log("Successful query!"); console.log(JSON.stringify(resp, null, 4)); }, function(err) { console.trace(err.message); }); |
There’s a few steps to dissect so let’s dive in step-by-step:
First we required the elasticsearch library because it gives us the library of functions that make it easy to access Elasticsearch.
Next we created a variable var client
which creates and stores our connection to Elasticsearch. From this point on we’ll use this client to do all our interactions with Elasticsearch.
We then use the search function on client
to create an aggregator.
We specify the index and type to perform the search on. The important part comes in the body where we define the aggregator by using the aggs
keyword. We gave our aggregator a name price_stats
that can be anything but choose something that makes sense. We specify the type of aggregator as extended_stats
.
* Lastly we specify what field the aggregator should evaluate the price
field for distinct values.
Here is how we ran the code which is in our working directory:
1 | $ node index.js |
And here was the response:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 | Successful query! { "took": 53, "timed_out": false, "_shards": { "total": 5, "successful": 5, "skipped": 0, "failed": 0 }, "hits": { "total": 14, "max_score": 0, "hits": [] }, "aggregations": { "price_stats": { "count": 14, "min": 0.99, "max": 5.59, "avg": 3.404285714285714, "sum": 47.66, "sum_of_squares": 192.80540000000002, "variance": 2.182653061224493, "std_deviation": 1.4773804727369633, "std_deviation_bounds": { "upper": 6.35904665975964, "lower": 0.4495247688117874 } } } } |
As you can see it gave us a multitude of stats on our price field including the average, maximum, and minimum that we were looking for.
Conclusion
In this tutorial we demonstrated how to use Elasticsearch aggregations to get an array of statitics on a field from our dataset using the convenient extended stats aggregation. There are many other things you can do with aggregation and you can consult the Elasticsearch documentation to learn more about it. The documentation is also useful if you need help with syntax. We hope you found this tutorial helpful and can apply it to your specific application. If you have questions or this didn’t work for you please reach out to us so we can help. Thank you.
Just the Code
If you’re already comfortable with NodeJS and aggregations here’s all the code we used to demonstrate how to do an extended stats aggregation in Elasticsearch using NodeJS.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | var elasticsearch = require("elasticsearch"); var client = new elasticsearch.Client({ hosts: ["http://localhost:9200"] }); /* Get Extended Stats Aggregation of price field */ client.search({ size: 0, index: 'store', type: 'products', body: { "aggs" : { "price_stats" : { "extended_stats" : { "field" : "price" } } } } }).then(function(resp) { console.log("Successful query!"); console.log(JSON.stringify(resp, null, 4)); }, function(err) { console.trace(err.message); }); |
Pilot the ObjectRocket Platform Free!
Try Fully-Managed CockroachDB, Elasticsearch, MongoDB, PostgreSQL (Beta) or Redis.
Get Started