Knowledgebase

Graylog server is an enterprise ready open source log management software suite. It collects logs from various sources and analyses them to discover and resolve issues. Graylog server is basically the combination of Elasticsearch, MongoDB and Graylog. Elasticsearch is a very popular open source application to store text and provide very powerful search capabilities. MongoDB is an open source application to store data in NoSQL format. Graylog collects logs from various sources and provides a web-based dashboard to manage and search through the logs. Graylog also provides a REST API for both configuration and data. It provides a configurable dashboard which can be used to visualize metrics and observe trends by using field statistics, quick values, and charts from one central location.

In this tutorial, you will learn to install Graylog Server on CentOS 7. This guide was written for Graylog Server 2.3, but may work on newer versions as well. You will also learn to install Java, Elasticsearch and MongoDB. We will also secure the MongoDB instance and set up an Nginx reverse proxy for the web-based dashboard and API.

Prerequisites

• A Rcs CentOS 7 server instance with at least 4GB RAM.
• A sudo user.

In this tutorial, we will use 192.0.2.1 as the public IP address of the server and graylog.example.com as the domain name pointed to the server. Replace all occurrences of 192.0.2.1 with your Rcs public IP address and graylog.example.com with your actual domain name.

Update your base system using the guide How to Update CentOS 7. Once your system has been updated, proceed to install Java.

Install Java

Elasticsearch requires Java 8 to run. It supports both Oracle Java and OpenJDK, but it is always recommended that you use Oracle Java when possible. Oracle provides ready to install RPM packages. Download the Oracle JDK RPM:

wget --no-cookies --no-check-certificate --header "Cookie:oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/8u144-b01/090f390dda5b47b9b721c7dfaa008135/jdk-8u144-linux-x64.rpm"

Install the RPM package.

sudo yum -y install jdk-8u144-linux-x64.rpm

If Java has installed successfully, then you should be able to verify its version.

java -version

You will see the following output.

[user@vultr ~]$ java -version
java version "1.8.0_144"
Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)

Set JAVA_HOME and JRE_HOME environment variable by running:

echo "export JAVA_HOME=/usr/java/jdk1.8.0_144/" >> ~/.bash_profile
echo "export JRE_HOME=/usr/java/jdk1.8.0_144/jre" >> ~/.bash_profile

Now, source the file using the following command.

source ~/.bash_profile

Run the echo $JAVA_HOME command to check if the environment variable is set or not.

[user@vultr ~]$ echo $JAVA_HOME
/usr/java/jdk1.8.0_144/

Install Elasticsearch

Elasticsearch is a distributed, real time, scalable and highly available application used to store the logs and search through them. It stores the data in indexes and searching through the data is very fast. It provides various sets of APIs, such as HTTP RESTful API and native Java API. Elasticsearch can be installed directly through the Elasticsearch repository. Create a new repository file for Elasticsearch.

sudo nano /etc/yum.repos.d/elasticsearch.repo

Populate the file with the following content.

[elasticsearch-5.x]
name=Elasticsearch repository for 5.x packages
baseurl=https://artifacts.elastic.co/packages/5.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

Import the PGP key used to sign the packages. This will ensure the integrity of the packages.

sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

Install the Elasticsearch package:

sudo yum -y install elasticsearch

Once the package is installed, open the Elasticsearch default configuration file.

sudo nano /etc/elasticsearch/elasticsearch.yml

Find the following line, uncomment it and change the value from my-application to graylog.

cluster.name: graylog

You can start Elasticsearch and enable it to automatically start at boot time:

sudo systemctl enable elasticsearch
sudo systemctl start elasticsearch

Elasticsearch is now running on port 9200. Verify that it is working properly by running:

curl -XGET 'localhost:9200/?pretty'

You should see output similar to the following.

[user@vultr ~]$ curl -XGET 'localhost:9200/?pretty'
{
  "name" : "-kYzFA9",
  "cluster_name" : "graylog",
  "cluster_uuid" : "T3JQKehzSqmLThlVkEKPKg",
  "version" : {
    "number" : "5.5.1",
    "build_hash" : "19c13d0",
    "build_date" : "2017-07-18T20:44:24.823Z",
    "build_snapshot" : false,
    "lucene_version" : "6.6.0"
  },
  "tagline" : "You Know, for Search"
}

If you encounter errors, wait for a few seconds and retry, as it takes time for Elasticsearch to complete its start-up process. Elasticsearch is now installed and working correctly.

Install MongoDB

MongoDB is free and open source NoSQL database server. Unlike traditional database which uses tables to organize their data, MongoDB is document-oriented and uses JSON-like documents without schemas. Graylog uses MongoDB to store its configuration and meta information. It can be installed directly through the MongoDB repository. Create a new repository file for MongoDB.

sudo nano /etc/yum.repos.d/mongodb.repo

Populate the file with the following content.

[mongodb-org-3.4]
name=MongoDB Repository
baseurl=https://repo.mongodb.org/yum/redhat/$releasever/mongodb-org/3.4/x86_64/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-3.4.asc

Install MongoDB by running:

sudo yum -y install mongodb-org

Start MongoDB server and enable it to start automatically.

sudo systemctl start mongod
sudo systemctl enable mongod

Install Graylog server

Download the latest repository for Graylog server.

sudo rpm -Uvh https://packages.graylog2.org/repo/packages/graylog-2.3-repository_latest.rpm
sudo yum -y update

Install Graylog by running:

sudo yum -y install graylog-server

Graylog server is now installed on your server. Before you can start it, you will need to configure a few things.

Configure Graylog

Install pwgen utility to generate strong passwords.

sudo yum -y install pwgen

Now generate a strong password secret.

pwgen -N 1 -s 96

You will output similar to:

[user@vultr ~]$ pwgen -N 1 -s 96
pJqhNbdEY9FtNBfFUtq20lG2m9daacmsZQr59FhyoA0Wu3XQyVZcu5FedPZ9eCiDfjdiYWfRcEQ7a36bVqxSyTzcMMx5Rz8v

Also, generate a 256-bit hash for the password of the root admin user:

echo -n StrongPassword | sha256sum

Replace StrongPassword with the password you wish to set for admin user. You will see:

[user@vultr ~]$ echo -n StrongPassword | sha256sum
05a181f00c157f70413d33701778a6ee7d2747ac18b9c0fbb8bd71a62dd7a223  -

Open the Graylog configuration file:

sudo nano /etc/graylog/server/server.conf

Find password_secret = , copy and paste the password generated through pwgen command. Find root_password_sha2 =, copy and paste the converted SHA 256-bit hash of your admin password. Find #root_email =, uncomment and provide your email address. Uncomment and set your time zone at root_timezone. For example:

password_secret = pJqhNbdEY9FtNBfFUtq20lG2m9daacmsZQr59FhyoA0Wu3XQyVZcu5FedPZ9eCiDfjdiYWfRcEQ7a36bVqxSyTzcMMx5Rz8v
root_password_sha2 = 05a181f00c157f70413d33701778a6ee7d2747ac18b9c0fbb8bd71a62dd7a223
root_email = mail@example.com
root_timezone = Asia/Kolkata

Enable the web-based Graylog interface by uncommenting #web_enable = false and setting the value to true. Also uncomment and change the following lines as specified.

rest_listen_uri = http://0.0.0.0:9000/api/
rest_transport_uri = http://45.76.214.19:9000/api/
web_enable = true
web_listen_uri = http://0.0.0.0:9000/

Save the file and exit from your text editor.

Restart the Graylog service by running:

sudo systemctl restart graylog-server

Configure Nginx as a reverse proxy

By default, the Graylog web interface listens to localhost on port 9000 and the API listens on port 9000 with URL /api. In this tutorial, we will use Nginx as the reverse proxy so that the application can be access via standard HTTP port. Install Nginx web server by running:

sudo yum -y install nginx

Open the default virtual host by typing.

sudo nano /etc/nginx/nginx.conf

Find the server block under http, and replace the whole server block with the following lines.

server
{
    listen 80 default_server;
    listen [::]:80 default_server ipv6only=on;
    server_name graylog.example.com 192.0.2.1;
    
    location / {
      proxy_set_header Host $http_host;
      proxy_set_header X-Forwarded-Host $host;
      proxy_set_header X-Forwarded-Server $host;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Graylog-Server-URL http://$server_name/api;
      proxy_pass       http://127.0.0.1:9000;
    }
}

Start Nginx and enable it to start automatically at boot time:

sudo systemctl start nginx
sudo systemctl enable nginx

Configure firewall and SELinux

If you are running a firewall on your server, you will need to configure the firewall to set an exception for certain ports. Allow Elasticsearch service and Nginx reverse proxy to connect from outside the network.

sudo firewall-cmd --zone=public --permanent --add-service=http
sudo firewall-cmd --zone=public --permanent --add-port=9200/tcp
sudo firewall-cmd --reload

If you have SELinux enabled on your system, then you will need to add a few exceptions in SELinux policies.

sudo setsebool -P httpd_can_network_connect 1
sudo semanage port -a -t http_port_t -p tcp 9000
sudo semanage port -a -t http_port_t -p tcp 9200
sudo semanage port -a -t mongod_port_t -p tcp 27017

Conclusion

The installation and basic configuration of Graylog server is now complete. You can now access the Graylog server on http://192.0.2.1 or http://graylog.example.com if you have DNS configured. Login using the username admin and the plain text version of the password you have set for root_password_sha2 earlier.

Congratulations - you have a fully working Graylog server installed on your CentOS 7 server.