Rabbitmq Install on Centos 7 in cluster mode – Legendshub Blog

As we are fully aware of importance of Rabbitmq in enterprise environements. Today we will discuss how can we install rabbitmq in cluster mode in AWS on Centos 7.

AWS Environment:

In order to start installation of rabbitmq we will launch two ec2 instances with Centos 7 on them. We will add autoscaling group and configure Load Balancer for them. Once we will have AWS environment set we are ready to start rabbitmq installation in cluster mode

Rabbitmq Install

We need to make sure we bring up the instances using the following role which enable them to describe ec2 instances and autoscaling groups.

{
"Version": "2012-10-17",
"Statement": [
              {
              "Effect": "Allow",
              "Action": [
                         "autoscaling:DescribeAutoScalingInstances",
                         "ec2:DescribeInstances"
                         ],
              "Resource": [
                           "*"
                           ]
              }
              ]
}

Now we will login into once instance and configure two repository one for Erlang installation and second for Rabbitmq installation.

cd /etc/yum.repos.d/ vi rabbit_erlang.repo [rabbitmq_erlang]
name=Repo to install erlang for centos7
baseurl=https://packagecloud.io/rabbitmq/erlang/el/7/$basearch
enabled=1
gpgcheck=0
repo_gpgcheck=1
gpgkey=https://packagecloud.io/rabbitmq/erlang/gpgkey
 metadata_expire=300

sslcacert=/etc/pki/tls/certs/ca-bundle.crt
sslverify=1 vi rabbitmq.repo [rabbitmq_erlang] name=Repo to install rabbitmq for centos7 baseurl=https://dl.bintray.com/rabbitmq/rpm/rabbitmq-server/v3.8.x/el/7/ enabled=1
gpgcheck=0

Now repo has been configured but we generally observed that package install may conflict with existing epel repositories in Centos. So its best practice to disable existent repositories and enable new one.

yum -q makecache -y --disablerepo='epel*' --enablerepo='rabbitmq*'

Once it is completed now we will go ahead and install rabbitmq-server , perl-JSON and erlang packages using yum command. Make sure latest version is installed

yum install erlang perl-JSON rabbitmq-server -y 
rpm -qa | grep rabbitmq

Now we have latest package of rabbitmq server installed on AWS instances. We will move ahead with maintaining hosts file for each node. To make sure every new node in existing auto scaling group join cluster successfully we need to find the way of automatic detection on AWS nodes within the autoscaling group. We can use the following script to do the same.

cat instance_discovery.sh
 !/bin/bash
curl http://169.254.169.254/latest/meta-data/instance-id -o "/etc/we.txt"
instance=$(cat /etc/we.txt)
group=$(aws ec2 describe-instances --instance-ids $instance --region us-east-1 | grep -B 1 autoscaling | grep Value | awk -F":" '{print $2}' | awk -F'"' '{print $2}')
for ID in $(aws autoscaling describe-auto-scaling-instances --region us-east-1 --query "AutoScalingInstances[?AutoScalingGroupName=='${group}'].InstanceId" --output text);
do
echo $ID
aws ec2 describe-instances --instance-ids $ID --region us-east-1 --query Reservations[].Instances[].PrivateIpAddress --output text >> /etc/ghost
done
for ID in $(aws autoscaling describe-auto-scaling-instances --region us-east-1 --query "AutoScalingInstances[?AutoScalingGroupName=='${group}'].InstanceId" --output text);
do
aws ec2 describe-instances --instance-ids $ID --region us-east-1 --query Reservations[].Instances[].PrivateDnsName --output text| awk -F"." '{print $1}' >> /etc/ghost1 done paste /etc/ghost /etc/ghost1 >> /etc/hosts rm -rf /etc/ghost /etc/ghost1 /etc/we.txt

Now once you run the above script all nodes within the autoscaling group will be amended to /etc/hosts file.

As we have hosts file thing managed now. We will enable aws peer discovery plugin with few more plugins. To know more about AWS Peer discovery plugin follow the link below or one can watch the video linked beneath.

https://www.rabbitmq.com/cluster-formation.html#peer-discovery-aws

rabbitmq-plugins enable rabbitmq_management
rabbitmq-plugins enable rabbitmq_federation
rabbitmq-plugins enable rabbitmq_federation_management
rabbitmq-plugins enable rabbitmq_auth_mechanism_ssl 
rabbitmq-plugins enable rabbitmq_management_agent
rabbitmq-plugins enable rabbitmq_shovel
rabbitmq-plugins enable rabbitmq_shovel_management
rabbitmq-plugins --offline enable rabbitmq_peer_discovery_aws

Now plugins are enabled we will now have to do configuration in rabbitmq file.As we will be using tags to add node in rabbitmq cluster.

vi /etc/rabbitmq/rabbitmq
cluster_formation.peer_discovery_backend = rabbitmq_peer_discovery_aws
cluster_formation.aws.region = us-east-1
cluster_formation.aws.instance_tags.region = us-east-1
cluster_formation.aws.instance_tags.service = rabbitmq

cluster_formation.aws.use_private_ip = true cluster_formation.aws.use_autoscaling_group = true log.file.level = debug

So we are ready to restart the service rabbitmq-server on first node.Once it is done we need to do package install on the second node. Do the config and restart the rabbitmq-service. It will automatically register the second node and you can see it registering by simultaneously running the following command.


Amrit Pal singh

Designing critical migration from legacy to cloud environments, extending devops approach into analytics, AI/ML to maximize the purpose of agile approach. We work to integrate compliance with risk, security and regulations, building seamless interfaces across multiple systems. I am fortunate to manage complex hierarchy of global portfolios, programs and projects and executive communication . Provide TCO analyses to low level architecture, design and implementation, readiness and building BCP/DR/HA, and around the clock monitoring & management using cloud tools relevant to our customer's situation and objectives.

Leave a Reply