rabbitmq,

Resolve RabbitMQ 'Node statistics not available' in Cluster Node

Nov 08, 2021 · 2 mins read · Post a comment

Setting up RabbitMQ cluster with master and slave instances can invoke some node destabilization like the error Node statistics not available. Here I’m going to show you few steps how can you resolve it.

Prerequisites

  • RabbitMQ

Solution

Step 1. The most common symptom is the 100% disk usage. So, SSH into the instance and check the status of the rabbitmq service.

systemctl status rabbitmq-server

If the status is Exited like below follow the Step 2.

● rabbitmq-server.service
   Loaded: loaded (/usr/lib/systemd/system/rabbitmq-server.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code)
  Process: 15630 ExecStart=/usr/sbin/rabbitmq-server (code=exited, status=1/FAILURE)
 Main PID: 15630 (code=exited, status=1/FAILURE)
   Status: "Exited."

Step 2. Check if /opt/rabbitmq directory has reached 100% usage.

df -h

If the output is similar like the following:

Filesystem                       Size  Used Avail Use% Mounted on
udev                             7,7G     0  7,7G   0% /dev
tmpfs                            1,6G  2,2M  1,6G   1% /run
/dev/mapper/datavg-lv_rabbitmq   3.0G  3.0G   34M 100% /opt/rabbitmq

Then you should find out which directory from /opt/rabbitmq is using the majority of the disk space.

du -sch /opt/rabbitmq/*

So if you figure out that the directory is old or contains messages that are not useful you can clean them up. Afterwards restart the RabbitMQ service, so it can be in a running status and fix the error.

systemctl status rabbitmq-server
...
● rabbitmq-server.service - RabbitMQ broker
   Loaded: loaded (/usr/lib/systemd/system/rabbitmq-server.service; enabled; vendor preset: disabled)
   Active: active (running)

Step 3. Even if the service status is running, it may still show the same message Node statistics not available. There is one more possibility. Check the /var/log/rabbitmq/*.log log files and see if the net tick timeout issue is popping up.

=INFO REPORT==== 1-Nov-2021::17:50:33 ===
node rabbit@server down: net_tick_timeout

To resolve this, open RabbitMQ default conf file, and update net_ticktime value, if not defined.

net_ticktime = 60

Restart the RabbitMQ service, so the changes can take effect and that’s it. The error should be gone.

systemctl restart rabbitmq-server

Conclusion

If there is something that you want to raise please do it in the comment section below. Feel free to leave a comment below and if you find this tutorial useful, follow our official channel on Telegram.

rabbitmq errors clusters