TOOLS | Balaji Bandi

Graylog 5.2 + OpenSearch 2.11 + MongoDB 6.0 All in One Server installation – PoC

November 19, 2023 / Balaji Bandi / 0 Comments

As part of my other old Installation of Graylog – Now i am Planning to install new Server with Graylog 5.2 it has lot new features and lot of fixes which help to build both Log Operation and Security SIEM Solution

You can refer old Blog Gray Log 4.3 – https://www.balajibandi.com/?p=1811

High Level Diagram :

Pre-Requisites :

Ubuntu 22.04 LTS
Update to latest with upgrade as on Today writing this Blog 18th Nov 2023

Installing MongoDB 6.0

#apt-get install gnupg curl (if you upgrade should be upto date as mine)

Import keys for MongoDB

#curl -fsSL https://pgp.mongodb.com/server-6.0.asc | \
   sudo gpg -o /usr/share/keyrings/mongodb-server-6.0.gpg \
   --dearmor

Create a list file for MongoDB

#echo "deb [ arch=amd64,arm64 signed-by=/usr/share/keyrings/mongodb-server-6.0.gpg ] https://repo.mongodb.org/apt/ubuntu jammy/mongodb-org/6.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-6.0.list

Update the packages list

#apt-get update

Install the latest stable version of MongoDB.

#apt-get install -y mongodb-org

Enable MongoDB during the operating system’s start up and verify it is running.

systemctl daemon-reload
systemctl enable mongod.service
systemctl restart mongod.service
systemctl --type=service --state=active | grep mongod

#systemctl status mongod

Now MongoDB successfully installed

Testing MongoDB

#mongosh

OpenSearch Installation.

Install the necessary packages

#apt-get update && apt-get -y install lsb-release ca-certificates curl gnupg2

Import the public GPG key. This key is used to verify that the APT repository is signed

#curl -o- https://artifacts.opensearch.org/publickeys/opensearch.pgp | sudo gpg --dearmor --batch --yes -o /usr/share/keyrings/opensearch-keyring

Create an APT repository for OpenSearch

#echo "deb [signed-by=/usr/share/keyrings/opensearch-keyring] https://artifacts.opensearch.org/releases/bundle/opensearch/2.x/apt stable main" | sudo tee /etc/apt/sources.list.d/opensearch-2.x.list

Update the APT and check OpenSearch available packages to install 

#apt-get update
#apt list -a opensearch

Installing Opensearch 2.11

#apt-get install opensearch

Move the default opensearch.yml to backup and replace with below config to start testing

vi opensearch.yml ( add below content)

cluster.name: graylog52
node.name: ${HOSTNAME}
path.data: /var/lib/opensearch
path.logs: /var/log/opensearch
discovery.type: single-node
network.host: 0.0.0.0
action.auto_create_index: false
plugins.security.disabled: true

:wq (save and quit)

Enable JVM options. (i used 4GB RAM as my Server have enough RAM)

#vi /etc/opensearch/jvm.options

-Xms4g
-Xmx4g

Configure the kernel parameters at runtime.

#sysctl -w vm.max_map_count=262144
#echo 'vm.max_map_count=262144' >> sudo /etc/sysctl.conf

Finally, enable the system service.

#systemctl daemon-reload
#systemctl enable opensearch.service
#systemctl start opensearch.service
#systemctl status opensearch.service

check the OpenSearch Initialized and working

#curl -X GET http://localhost:9200 -u ‘admin:admin’

check OpenSearch Plugin installed part of OpenSearch Installation

#curl -X GET http://localhost:9200/_cat/plugins?v -u ‘admin:admin’

you can also list Plugin executing below command :

#/usr/share/opensearch/bin/opensearch-plugin list

Now Installing GrayLog 5.2

#wget https://packages.graylog2.org/repo/packages/graylog-5.2-repository_latest.deb
#dpkg -i graylog-5.2-repository_latest.deb
#apt-get update && sudo apt-get install graylog-server 

Now the GrayLog Server installed generate password secret and password sha2

Edit the Configuration File
Read the instructions within the configurations file and edit as needed, located at /etc/graylog/server/server.conf. Additionally, add password_secret and root_password_sha2 as these are mandatory and Graylog will not start without them.

To create your password_secret run the following command:

#pwgen -N 1 -s 96
To create your root_password_sha2 run the following command:

#echo -n "Enter Password: " && head -1 </dev/stdin | tr -d '\n' | sha256sum | cut -d" " -f1

Point to ElasticSearch config to OpenSearch 

Configure the Connection to your OpenSearch :

elasticsearch_hosts = http://hostname:9200

So high level graylog config Looks like below :

cat /etc/graylog/server/server.conf         | egrep -v "^\s*(#|$)"
is_leader = true
node_id_file = /etc/graylog/server/node-id
password_secret = YYYYYYYYYYYYYY
root_password_sha2 = XXXXXXXXXXXXX
bin_dir = /usr/share/graylog-server/bin
data_dir = /var/lib/graylog-server
plugin_dir = /usr/share/graylog-server/plugin
http_bind_address = 192.168.1.238:9000
stream_aware_field_types=false
elasticsearch_hosts = http://pocsyslog:9200
disabled_retention_strategies = none
allow_leading_wildcard_searches = false
allow_highlighting = false
output_batch_size = 500
output_flush_interval = 1
output_fault_count_threshold = 5
output_fault_penalty_seconds = 30
processbuffer_processors = 5
outputbuffer_processors = 3
processor_wait_strategy = blocking
ring_size = 65536
inputbuffer_ring_size = 65536
inputbuffer_processors = 2
inputbuffer_wait_strategy = blocking
message_journal_enabled = true
message_journal_dir = /var/lib/graylog-server/journal
lb_recognition_period_seconds = 3
mongodb_uri = mongodb://localhost/graylog
mongodb_max_connections = 1000

Start the GrayLog Service :

#systemctl daemon-reload
#systemctl enable graylog-server.service
#systemctl start graylog-server.service
#systemctl --type=service --state=active | grep graylog


Status :