How To Return All Documents From An Index In Elasticsearch

Introduction

• You can use cURL in a UNIX terminal or Windows command prompt, the Kibana Console UI, or any one of the various low-level clients available to make an API call to get all of the documents in an Elasticsearch index.

• All of these methods use a variation of the GET request to search the index.

The Kibana Console UI Method

• If the Kibana service is running on your server you can navigate to the Kibana interface in a browser at http://localhost:5601 or http://{YOUR_DOMAIN}:5601.

• Click on Dev Tools and open the UI console. Once there you can type GET {YOUR_INDEX} and click the green arrow to get a response:

• This request will verify that the index exists—just make sure to replace {YOUR_INDEX} with the actual name of your Elasticsearch index that you’d like to query.

• Make another GET request with the _search API to return all of the documents in an index using a "match_all" query:

  1 2 3 4 5 6

  GET {YOUR_INDEX}/_search {     "query": {         "match_all": {}     } }

• You can also find all documents with a particular tag across several indices. The example below shows how to find the documents with the tag "nice" after cross-referencing the indices "pets" and "flowers":

  1	GET /pets,flowers/_search?q=tag:nice

• Another useful tool is the _all parameter. Use this option to search for all documents with the "nice" tag across all available indices:

  1	GET /_all/_search?q=tag:nice

The Curl Method

The cURL must first be installed and running before you can make HTTP requests to an Elasticsearch index. Most UNIX-based operating systems like macOS or Linux typically come with the cURL library installed however. The package library can be downloaded from the cURL Project’s website.

• The cURL way of making an HTTP request is a bit more complicated than the Kibana Console method, although it follow the same basic principle, and the body of the request is basically the same.

• The header of an Elasticsearch cURL request uses the -X option, as well as some others, and must include the domain host of the Elasticsearch server.

• Structure the request header like this to get the same "hits" as the above Kibana requests:

  1	curl -X GET "https://{YOUR_SERVER}:9200/{YOUR_INDEX}_search" -H 'Content-Type: application/json' -d

• As you can see, the cURL header has a few options (like -H and -d) that Kibana doesn’t have.

• Open a command prompt or terminal window on your machine, and make another GET request, like in the following example, to have cURL return all of your index’s documents:

  1 2 3 4 5 6 7

  curl -X GET "localhost:9200/animals/_search?pretty" -H 'Content-Type: application/json' -d' {     "query": {         "match_all": {}     } } '

Use the ?pretty option to have the request return the JSON objects in a more human-readable format.

• Don’t forget to enclose the body of the request in a single quotation mark (') to avoid any errors.

• The next example performs a scan search on the default Elasticsearch port of 9200 to matches all available documents:

This "scroll" option will limit the results of the search.

NOTE: Elasticsearch now enforces a strict content-type checking for its cURL requests since version 6.0 of the Elastic Stack. This means that a cURL request must now have -H 'Content-Type: application/json' as a header option whenever the request has a JSON object in its content body in order to explicitly specify that the content type is in JSON format. If you omit this in the header you’ll get a 406 Content-Type error:

Use the command curl --help for more information about the various options.

Low-Level Clients:

• There are many low-level clients for different programming languages one can use to get Elasticsearch documents. Each respective client has its own unique syntax, of course, but they all map fairly consistently to the same basic API requests as the ones in this article.

• They are all essentially light “wrappers” for the native Java client, and the Get API is just one of the many APIs accessible using their low-level clients. Check out Elastic’s documentation for a complete list of all the Elasticsearch Clients.

Conclusion

Use GET requests, the Get API, and the _search API to query documents in an Elasticsearch index.