Daemon.err mqttEv[2031]: mqttEvRdReady: mosquitto [[gwBridgeS Mqtt Client]] loop misc ERR: 4

Issue: A power cycle after a new SSL certificate on the MQTT broker does no longer connect MQTT for ChirpStack 3.x (JSON) and fills the log with hundrets of entries per second containing: daemon.err mqttEv[2031]: mqttEvRdReady: mosquitto [[gwBridgeS Mqtt Client]] loop misc ERR: 4

Setup:
RAK 7258
Firmware Version: 1.3.2_RAK b75
Region: EU863-870
Protocol: MQTT for ChirpStack 3.x (JSON)
SSL/TLS Mode: CA signed server certificate
TLS Version: TLSv1.2

LoRa® Server: ChirpStack EU868 @ 3.13.0

Details:
This only occurs after the MQTT broker certificate is updated, and not immediatelly because after reconnecting to the MQTT broker with the new certificate it keeps running fine. Only after a power cycle of the RAK7258 does it no longer connect and is the log filled up with:
daemon.err mqttEv[2031]: mqttEvRdReady: mosquitto [[gwBridgeS Mqtt Client]] loop misc ERR: 4

The above occurs hundrets of times per second, every now and then with this variant:
daemon.err mqttEv[2031]: mqttLoopMiscTimerHandler: mosquitto [[gwBridgeS Mqtt Client]] loop misc ERR: 4

As we only get a new certificate every 3 months (Let’s Encrypt) it is difficult to troubleshoot. I first noticed it 3 months ago and then tought it was related to: Gateways using MQTT / SSL with CA server signed cert from Lets Encrypt is broken
and the fact that we shortly had a certificate that doesn’t work with the old OpenSSL used by the RAK7258.

However we now have changed the Let’s Encrypt certificate generation so it is immediatelly correct, still the problem occured after we got a new certificate today.

The problem can be resolved by going to Lora Network > Network Settings and then press Save & Apply. Last time this resolved it immediately. Today it initially resolved it, but after a power cycle it occured again. Again pressed Save & Apply, got all kind of other errors about failing to start the gateway (forgot to capture those), but after another power cycle it was working again. It keeps on working now after several additional power cycles.

I have a RAK7258 I use for testing that is power cycled every day, it worked without issue for 3 months until the certificate was updated.

We have several RAK7258 in the field, they are still running ok, last time they disconnected only after they got power cycled days/weeks after the certificate update and we had to rush over to resolve it. This time we will visit them preemptive after the new certificate, but that is not a workable situation.

3 Likes

Same issues here with RAK7268CV2, WisGateOS 2.1.2 and a regular certificate that has been renewed. After reboot we get an infinite loop of “mqttEv[3011]: mqttEvRdReady: mosquitto [[AppSrv Mqtt Client]] loop misc ERR: 4”. Since we have a lot of devices in the field, we would like to see a self discovery on this issues.

I have the same setup as you @nkoot, except that it’s a RAK7240. My company switched to Lets Encrypt certificates on the server and the gateway disonnected after the change. In my case, it doesn’t reconnect after I reboot it. Please comment if you have found a permanent solution or more details about the error.

Same issue here. Any solution?

Same here, any solution?