Graylog 5.2 + OpenSearch 2.11 + MongoDB 6.0 All in One Server installation – PoC
As part of my other old Installation of Graylog – Now i am Planning to install new Server with Graylog 5.2 it has lot new features and lot of fixes which help to build both Log Operation and Security SIEM Solution
You can refer old Blog Gray Log 4.3 – https://www.balajibandi.com/?p=1811
High Level Diagram :
Pre-Requisites :
- Ubuntu 22.04 LTS
- Update to latest with upgrade as on Today writing this Blog 18th Nov 2023
Installing MongoDB 6.0
#apt-get install gnupg curl (if you upgrade should be upto date as mine)
Import keys for MongoDB
#curl -fsSL https://pgp.mongodb.com/server-6.0.asc | \
sudo gpg -o /usr/share/keyrings/mongodb-server-6.0.gpg \
--dearmor
Create a list file for MongoDB
#echo "deb [ arch=amd64,arm64 signed-by=/usr/share/keyrings/mongodb-server-6.0.gpg ] https://repo.mongodb.org/apt/ubuntu jammy/mongodb-org/6.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-6.0.list
Update the packages list
#apt-get update
Install the latest stable version of MongoDB.
#apt-get install -y mongodb-org
Enable MongoDB during the operating system’s start up and verify it is running.
systemctl daemon-reload
systemctl enable mongod.service
systemctl restart mongod.service
systemctl --type=service --state=active | grep mongod
#systemctl status mongod
Now MongoDB successfully installed
Testing MongoDB
#mongosh
OpenSearch Installation.
Install the necessary packages
#apt-get update && apt-get -y install lsb-release ca-certificates curl gnupg2
Import the public GPG key. This key is used to verify that the APT repository is signed
#curl -o- https://artifacts.opensearch.org/publickeys/opensearch.pgp | sudo gpg --dearmor --batch --yes -o /usr/share/keyrings/opensearch-keyring
Create an APT repository for OpenSearch
#echo "deb [signed-by=/usr/share/keyrings/opensearch-keyring] https://artifacts.opensearch.org/releases/bundle/opensearch/2.x/apt stable main" | sudo tee /etc/apt/sources.list.d/opensearch-2.x.list
Update the APT and check OpenSearch available packages to install
#apt-get update
#apt list -a opensearch
Installing Opensearch 2.11
#apt-get install opensearch
Move the default opensearch.yml to backup and replace with below config to start testing
vi opensearch.yml ( add below content)
cluster.name: graylog52
node.name: ${HOSTNAME}
path.data: /var/lib/opensearch
path.logs: /var/log/opensearch
discovery.type: single-node
network.host: 0.0.0.0
action.auto_create_index: false
plugins.security.disabled: true
:wq (save and quit)
Enable JVM options. (i used 4GB RAM as my Server have enough RAM)
#vi /etc/opensearch/jvm.options
-Xms4g
-Xmx4g
Configure the kernel parameters at runtime.
#sysctl -w vm.max_map_count=262144
#echo 'vm.max_map_count=262144' >> sudo /etc/sysctl.conf
Finally, enable the system service.
#systemctl daemon-reload
#systemctl enable opensearch.service
#systemctl start opensearch.service
#systemctl status opensearch.service
check the OpenSearch Initialized and working
#curl -X GET http://localhost:9200 -u ‘admin:admin’
check OpenSearch Plugin installed part of OpenSearch Installation
#curl -X GET http://localhost:9200/_cat/plugins?v -u ‘admin:admin’
you can also list Plugin executing below command :
#/usr/share/opensearch/bin/opensearch-plugin list
Now Installing GrayLog 5.2
#wget https://packages.graylog2.org/repo/packages/graylog-5.2-repository_latest.deb
#dpkg -i graylog-5.2-repository_latest.deb
#apt-get update && sudo apt-get install graylog-server
Now the GrayLog Server installed generate password secret and password sha2
Edit the Configuration File
Read the instructions within the configurations file and edit as needed, located at /etc/graylog/server/server.conf. Additionally, add password_secret and root_password_sha2 as these are mandatory and Graylog will not start without them.
To create your password_secret run the following command:
#pwgen -N 1 -s 96
To create your root_password_sha2 run the following command:
#echo -n "Enter Password: " && head -1 </dev/stdin | tr -d '\n' | sha256sum | cut -d" " -f1
Point to ElasticSearch config to OpenSearch
Configure the Connection to your OpenSearch :
elasticsearch_hosts = http://hostname:9200
So high level graylog config Looks like below :
cat /etc/graylog/server/server.conf | egrep -v "^\s*(#|$)"
is_leader = true
node_id_file = /etc/graylog/server/node-id
password_secret = YYYYYYYYYYYYYY
root_password_sha2 = XXXXXXXXXXXXX
bin_dir = /usr/share/graylog-server/bin
data_dir = /var/lib/graylog-server
plugin_dir = /usr/share/graylog-server/plugin
http_bind_address = 192.168.1.238:9000
stream_aware_field_types=false
elasticsearch_hosts = http://pocsyslog:9200
disabled_retention_strategies = none
allow_leading_wildcard_searches = false
allow_highlighting = false
output_batch_size = 500
output_flush_interval = 1
output_fault_count_threshold = 5
output_fault_penalty_seconds = 30
processbuffer_processors = 5
outputbuffer_processors = 3
processor_wait_strategy = blocking
ring_size = 65536
inputbuffer_ring_size = 65536
inputbuffer_processors = 2
inputbuffer_wait_strategy = blocking
message_journal_enabled = true
message_journal_dir = /var/lib/graylog-server/journal
lb_recognition_period_seconds = 3
mongodb_uri = mongodb://localhost/graylog
mongodb_max_connections = 1000
Start the GrayLog Service :
#systemctl daemon-reload
#systemctl enable graylog-server.service
#systemctl start graylog-server.service
#systemctl --type=service --state=active | grep graylog
Status :
Now Graylog running – check the GUI :
http://192.168.1.238:9000
If all good you see welcome page as below :
You can check the OpenSearch health :
System –> indicies
This shows Graylog 5.2 and OpenSearch 2.11 working as expected.
Now i will process some Logs to GrayLog and test it.
Just added Router to Test Lab and configured the Syslog pointing to GrayLog 5.2 we can see the Logs in GUI :
Happy Logging..!!!!!!!!!