How to Configure Logstash Monitoring in Elasticsearch

Introduction

Logstash is a plugin tool that provides a mechanism for collecting data regarding metric parameters established by the user. There are a significant number of details that the program can scan for and retrieve. Below is a walkthrough of the basic steps to take for arranging directives to target a specific set of variables. The process outlined here will enable this tool to provide feedback. Following these instructions and tips will ensure the info coming in contains the correct elements.

Prerequisites

• Elasticsearch, Logstash, and JDK 8 (Logstash does not yet officially support JDK 9) installed.

• Elasticsearch is dependent and built-upon Java. To make sure Java is installed, and what version is running, run this command:

• If Java and JDK 8 are installed properly on the terminal, it should look like this:

• It is recommended to install Java version 1.8 or later.

• (Github forum for Logstash compatibility issues and JDK 9+)

• If still having issues with Java Runtime Environment (JRE), check that the JAVA_HOME variable for the JDK environment file has been set correctly.

Configuring Logstash

• To configure Logstash, create a config file that specifies which plugins to use and settings for each plugin. Event fields can be referred to in configuration and conditionals are used to process events of a certain criteria. When running logstash, use the -f to specify the config file. F__

• To walk through creating a basic config file and running Logstash with it, create a file named "logstash-simple.conf," then save in the same Logstash directory:

• Run Logstash in terminal and specify the file as -f flag:

• Logstash reads the config file and sends output to both Elasticsearch and stdout. Before moving on to more complex examples, here is a more detailed look at the structure of a config file:

A Logstash config file has separate sections for plugin types added to the event processing pipeline. For example:

{ `} filter { `} output { `}

The logstash.yml Configuration File

• When Logstash runs it will automatically gather metrics to evaluate cluster health and performance. Elasticsearch and Logstash use X-Pack to control monitored data collection from nodes.

Add relevant X-Pack directives to the Logstash config file (/etc/logstash/logstash.yml):

WARNING: The xpack.monitoring service has been depreciated in Elasticsearch 7.x.

• The Logstash YAML config file should be found in the main install directory for Logstash. For Linux this should be:

• Likewise, the Elastisearch YAML config file (elasticsearch.yml) is found in the main directory of the Elasticsearch installation, and the layout of the default directory structure will depend upon the system and package version. In Linux, this file will be located in the /etc/ directory.

X-Pack Monitoring Settings

• If running the Elasticstack on Linux, it can be edited by using nano editor:

• The Security and Monitoring services for X-Pack can be toggled on/off using the following boolean values:

• These will likely not be present in the YAML file, because the services run by default. If this is true, add them and save.

• For a list of all X-Pack config settings and features refer to Elastic’s website.

Conclusion

Making use of the array of valuable tools with this service allows a user to finely tune their analytics. By providing the necessary data it becomes possible to hone the qualities of the designated sources to perform as expected. The capacity of the structure a user creates with the Elasticsearch service can be greatly enhanced in performance with the available options. Once the matrix is designed as preferred the automated tools will do the work. Therefore, take advantage of the simplified directions stated here to experience the power of choice integrated into their impressive add-ons.