Build an Elasticsearch Web Application in Python (Part 1)

Introduction to Elasticsearch web applications in Python

Web applications are a great way to bring your Elasticsearch cluster’s search functionality to the front end for a better user experience. This article will demonstrate how to utilize Elasticsearch’s efficient search and analytics engine in a simple web app that can execute complex queries of its document data.

Build an Elasticsearch web app in Python using the Bottle framework

This two-part article series will demonstrate how you can create a simple Python web app using the open-source, micro web-framework called Bottle. Bottle is a WSGI-compliant server that is more than capable of creating a RESTful Elasticsearch web server.

By the end of the series you’ll be able to create a web application that will allow users to retrieve Elasticsearch documents on the front end by utilizing Python’s Elasticsearch client and the Bottle framework, on the back end.

Prerequisites to using the Python Bottle framework with Elasticsearch

The example code in this article has been tested on Python 3, and assumes that you can use the PIP3 package manager for Python 3 to install the necessary package libraries for Bottle and Elasticsearch.

Use PIP3 to install and upgrade the Python modules

The following command will use the python3 -m pip command to install and upgrade the low-level Python client for Elasticsearch:

You can just use pip3 install elasticsearch to install the client as well. Here’s the same command to install the Bottle framework:

Access the bottle.__version__ attribute to check which version of the Bottle framework is installed, and use elasticsearch.__version__ for Elasticsearch.

NOTE: Since version 0.13 of Bottle there is no longer any support for Python versions that are 2.6 or older.

Create a project directory for the Elasticsearch web app

Navigate to your project directory, or use the mkdir command in a terminal window to create one. The relative path of your project to your web server’s root directory is irrelevant because Bottle will use a port and routes for the app (similar to Node.JS).

Create a Python script for the Elasticsearch web app

Change into the directory (using cd), and then create a new Python script and save it inside of the project directory folder. Use an IDE with Python support to create a new Python script for the Elasticsearch web app.

If you’re using Visual Studio Code use the code command (e.g. code elastic_app.py), or use the subl command for Sublime.

Import the Python packages for Elasticsearch and Bottle

Import the run and get libraries for the web application, as well as the Elasticsearch low-level client library using the following Python code:

We’re going to use Python’s built-in json library, as well, to format the JSON dict responses returned by the Elasticsearch cluster.

Declare globals for the Bottle and Elasticsearch servers

Chose a port between 1 and 65535 for the Bottle web server and create a global integer object for it (or hard-code the value directly to the run() method’s port parameter).

The following code creates globals for your server’s domain, and for the Bottle server and Elasticsearch cluster’s ports:

NOTE: Python may return a unclosed <socket.socket> error if there’s an http protocol included in the domain string. The default port for Elasticsearch is 9200, but this can be configured in your Elasticsearch installation’s elasticsearch.yml file, and it will take affect after you restart the service.

Get a client instance of the Elasticsearch library

Now we’re going to attempt to connect to Elasticsearch’s cluster by passing the domain string to the Elasticsearch client library’s Elasticsearch() method.

Here’s some code that will do this inside of a try-except indentation block:

If the cluster isn’t running, or if one of your global parameters declared earlier are invalid, it will set the client instance to None.

Create a web server using Bottle

The following example code using a simple GET HTTP protocol (using the @get() routing decorator) that will return the Elasticsearch cluster information to the front end:

The above code defines a Python function that will return the HTML to the front end browser when it’s called.

Check if the Elasticsearch client is valid while concatenating the HTML string

If the client object instance of Elasticsearch is not valid (i.e. set to None), then we’ll return an HTML response warning the user by passing a message inside of an <h3> header tag:

Return the Elasticsearch client information using the Bottle framework

If the client object instance is valid, then use the following code to concatenate an HTML string, and append the Elasticsearch cluster information to the string:

NOTE: The above code uses a little JavaScript to “stringify” the JSON response returned by the Elasticsearch API call, and then changes the <h3> element’s inner HTML to reflect the cluster information. Likewise, the json.dumps() method converts a Python dict object into a language-agnostic, JSON-compliant string.

Return the HTML string at the end of the function for the GET request

Conclude the Bottle framework’s GET function by having it return the concatenated HTML string:

Run the Bottle server for the Elasticsearch web app

The last step is to pass the globals (declared above) to the Bottle library’s run() method to have Python run the server on the port specified:

Execute the Python script using the python3 command, and then navigate to localhost:1234 (or 127.0.0.1:1234) in a browser tab, and the Python framework should return your Elasticsearch cluster information to you in a web page.

You should see Python return something like the following in your terminal window:

NOTE: If Python returns an error saying: OSError: [Errno 48] Address already in use, then input lsof -ti tcp:1234 to grep for the offending process using the open port. Once you get the PID for the process you can use the kill -9 command (followed by the PID) to kill the process before trying to run the script again.

Conclusion to creating an Elasticsearch web application in Python

This article concludes the first part of this Elasticsearch and Bottle web application tutorial. The information and code contained here should help you to get started in building a fully capable, Elasticsearch web application in Python that can potentially query documents in an index from a web page.

Just the Code