Build an Elasticsearch Web Application in Python (Part 1)

Introduction to Elasticsearch web applications in Python

Web applications are a great way to bring your Elasticsearch cluster’s search functionality to the front end for a better user experience. This article will demonstrate how to utilize Elasticsearch’s efficient search and analytics engine in a simple web app that can execute complex queries of its document data.

Build an Elasticsearch web app in Python using the Bottle framework

This two-part article series will demonstrate how you can create a simple Python web app using the open-source, micro web-framework called Bottle. Bottle is a WSGI-compliant server that is more than capable of creating a RESTful Elasticsearch web server.

By the end of the series you’ll be able to create a web application that will allow users to retrieve Elasticsearch documents on the front end by utilizing Python’s Elasticsearch client and the Bottle framework, on the back end.

Prerequisites to using the Python Bottle framework with Elasticsearch

The example code in this article has been tested on Python 3, and assumes that you can use the PIP3 package manager for Python 3 to install the necessary package libraries for Bottle and Elasticsearch.

Use PIP3 to install and upgrade the Python modules

The following command will use the python3 -m pip command to install and upgrade the low-level Python client for Elasticsearch:

sudo python3 -m pip install --upgrade elasticsearch

You can just use pip3 install elasticsearch to install the client as well. Here’s the same command to install the Bottle framework:

sudo python3 -m pip install --upgrade bottle

Access the bottle.__version__ attribute to check which version of the Bottle framework is installed, and use elasticsearch.__version__ for Elasticsearch.

Screenshot of Python 3.7 IDLE returning the version numbers for Bottle and Elasticsearch

NOTE: Since version 0.13 of Bottle there is no longer any support for Python versions that are 2.6 or older.

Create a project directory for the Elasticsearch web app

Navigate to your project directory, or use the mkdir command in a terminal window to create one. The relative path of your project to your web server’s root directory is irrelevant because Bottle will use a port and routes for the app (similar to Node.JS).

Create a Python script for the Elasticsearch web app

Change into the directory (using cd), and then create a new Python script and save it inside of the project directory folder. Use an IDE with Python support to create a new Python script for the Elasticsearch web app.

If you’re using Visual Studio Code use the code command (e.g. code elastic_app.py), or use the subl command for Sublime.

Import the Python packages for Elasticsearch and Bottle

Import the run and get libraries for the web application, as well as the Elasticsearch low-level client library using the following Python code:

# import the run and get methods from the bottle library
from bottle import run, get

# import the elasticsearch client library
from elasticsearch import Elasticsearch

# import Python's json library to format JSON responses
import json

We’re going to use Python’s built-in json library, as well, to format the JSON dict responses returned by the Elasticsearch cluster.

Declare globals for the Bottle and Elasticsearch servers

Chose a port between 1 and 65535 for the Bottle web server and create a global integer object for it (or hard-code the value directly to the run() method’s port parameter).

The following code creates globals for your server’s domain, and for the Bottle server and Elasticsearch cluster’s ports:

# ResourceWarning: unclosed <socket.socket> error if HTTP in domain
DOMAIN = "localhost"
ELASTIC_PORT = 9200
BOTTLE_PORT = 1234

NOTE: Python may return a unclosed <socket.socket> error if there’s an http protocol included in the domain string. The default port for Elasticsearch is 9200, but this can be configured in your Elasticsearch installation’s elasticsearch.yml file, and it will take affect after you restart the service.

Get a client instance of the Elasticsearch library

Now we’re going to attempt to connect to Elasticsearch’s cluster by passing the domain string to the Elasticsearch client library’s Elasticsearch() method.

Here’s some code that will do this inside of a try-except indentation block:

try:

    # concatenate a string for the Elasticsearch connection
    domain_str = DOMAIN + ":" + str(ELASTIC_PORT)

    # declare a client instance of the Python Elasticsearch library
    client = Elasticsearch( domain_str )

    # info() method raises error if domain or conn is invalid
    print (json.dumps( Elasticsearch.info(client), indent=4 ), "\n")

except Exception as err:
    print ("Elasticsearch() ERROR:", err, "\n")

    # client is set to none if connection is invalid
    client = None

If the cluster isn’t running, or if one of your global parameters declared earlier are invalid, it will set the client instance to None.

Create a web server using Bottle

The following example code using a simple GET HTTP protocol (using the @get() routing decorator) that will return the Elasticsearch cluster information to the front end:

@get('/')
def elastic_app():

The above code defines a Python function that will return the HTML to the front end browser when it’s called.

Check if the Elasticsearch client is valid while concatenating the HTML string

If the client object instance of Elasticsearch is not valid (i.e. set to None), then we’ll return an HTML response warning the user by passing a message inside of an <h3> header tag:

    # call the func to return HTML to framework
    if client == None:
        html = '<h3 style="color:red">Warning: Elasticsearch cluster is not running on'
        html += ' port: ' + str(ELASTIC_PORT) + '</h3>'

Return the Elasticsearch client information using the Bottle framework

If the client object instance is valid, then use the following code to concatenate an HTML string, and append the Elasticsearch cluster information to the string:

    else:
        # get the Elasticsearch client info
        client_info = Elasticsearch.info(client)

        # return the client data as JSON object to front end
        html = "<h2>Elasticsearch client information:</h2><br>"
        html += '<h3 id="elastic-resp"></h3><br>'
        html += '<script>'
        html += 'var respEle = document.getElementById("elastic-resp");'
        html += 'var elasticResp = JSON.stringify(' + json.dumps(client_info)
        html += ', null, 4);'
        html += 'respEle.innerHTML = elasticResp;'
        html += '</script>'

NOTE: The above code uses a little JavaScript to “stringify” the JSON response returned by the Elasticsearch API call, and then changes the <h3> element’s inner HTML to reflect the cluster information. Likewise, the json.dumps() method converts a Python dict object into a language-agnostic, JSON-compliant string.

Return the HTML string at the end of the function for the GET request

Conclude the Bottle framework’s GET function by having it return the concatenated HTML string:

    # return the HTML data to the front end
    return html

Run the Bottle server for the Elasticsearch web app

The last step is to pass the globals (declared above) to the Bottle library’s run() method to have Python run the server on the port specified:

# pass a port for the framework's server
run(
    host = DOMAIN,
    port = BOTTLE_PORT,
    debug = True
)

Execute the Python script using the python3 command, and then navigate to localhost:1234 (or 127.0.0.1:1234) in a browser tab, and the Python framework should return your Elasticsearch cluster information to you in a web page.

You should see Python return something like the following in your terminal window:

Listening on http://localhost:1234/
Hit Ctrl-C to quit.

Screenshot of the Python Bottle framwork returning Elasticsearch cluster information in browser tab

NOTE: If Python returns an error saying: OSError: [Errno 48] Address already in use, then input lsof -ti tcp:1234 to grep for the offending process using the open port. Once you get the PID for the process you can use the kill -9 command (followed by the PID) to kill the process before trying to run the script again.

Conclusion to creating an Elasticsearch web application in Python

This article concludes the first part of this Elasticsearch and Bottle web application tutorial. The information and code contained here should help you to get started in building a fully capable, Elasticsearch web application in Python that can potentially query documents in an index from a web page.

Just the Code

#!/usr/bin/env python3
#-*- coding: utf-8 -*-

# import the run and get methods from the bottle library
from bottle import run, get

# import the elasticsearch client library
from elasticsearch import Elasticsearch

# import Python's json library to format JSON responses
import json

# globals for the Elasticsearch domain
# ResourceWarning: unclosed <socket.socket> error if HTTP in domain
DOMAIN = "localhost"
ELASTIC_PORT = 9200
BOTTLE_PORT = 1234

try:

    # concatenate a string for the Elasticsearch connection
    domain_str = DOMAIN + ":" + str(ELASTIC_PORT)

    # declare a client instance of the Python Elasticsearch library
    client = Elasticsearch( domain_str )

    # info() method raises error if domain or conn is invalid
    print (json.dumps( Elasticsearch.info(client), indent=4 ), "\n")

except Exception as err:
    print ("Elasticsearch() ERROR:", err, "\n")

    # client is set to none if connection is invalid
    client = None

@get('/')
def elastic_app():

    # call the func to return HTML to framework
    if client == None:
        html = '<h3 style="color:red">Warning: Elasticsearch cluster is not running on'
        html += ' port: ' + str(ELASTIC_PORT) + '</h3>'
    else:
        # get the Elasticsearch client info
        client_info = Elasticsearch.info(client)

        # return the client data as JSON object to front end
        html = "<h2>Elasticsearch client information:</h2><br>"
        html += '<h3 id="elastic-resp"></h3><br>'
        html += '<script>'
        html += 'var respEle = document.getElementById("elastic-resp");'
        html += 'var elasticResp = JSON.stringify(' + json.dumps(client_info)
        html += ', null, 4);'
        html += 'respEle.innerHTML = elasticResp;'
        html += '</script>'

    # return the HTML data to the front end
    return html

# pass a port for the framework's server
run(
    host = DOMAIN,
    port = BOTTLE_PORT,
    debug = True
)

Pilot the ObjectRocket Platform Free!

Try Fully-Managed CockroachDB, Elasticsearch, MongoDB, PostgreSQL (Beta) or Redis.

Get Started

Keep in the know!

Subscribe to our emails and we’ll let you know what’s going on at ObjectRocket. We hate spam and make it easy to unsubscribe.