Receiving RAK7249 LoRaServer MQTT feed with Arduino

LPWAN Industrial Gateways / Concentrators RAK7249
#21

This is mistaken, and may be the source of your confusion

The MQTT bridge is a component that sits between the packet forwarder and the MQTT broker which the built in server uses to communicate with the packet forwarder aspect of the gateway itself.

It has no role for the decoded application data.

The actual flow is more like this:

concentrator chip
packet forwarder
legacy UDP protocol
MQTT bridge
MQTT broker for gateway traffic
Built in Server
MQTT broker for application traffic

It’s that second MQTT broker (external or internal) that you need to point both the server and the Arduino at so that the decoded application traffic (rather than raw encoded gateway traffic) can flow between them.

1 Like
#22

Well, progress. Thank you for your help.

I decided to try to get the RAK7249 going using the built-in server and the MQTT bridge working to an MQTT on my laptop rather then a RPi MQTT.

I have managed to get the MQTT on the laptop to connect to the 7249. So pretty excited about that.

I have also subscribed to a series of topics associated with nodes that the gateway can see.

So the issue now is, I dont get to see any traffic/data that I have subscribed to.

One of the nodes is the rak7204 which hasn’t been “seen” for 3 months, so I may be a bit overly hopeful. However one is DHT22 temp/humidity, and I can see that on the TNN when configured.

I’m thinking I’ll start again with a new node and get that to go afresh.

Any ideas, anyone?

#23

That’s a mistake. You’re still confusing the MQTT bridge’s task of feeding encrypted raw traffic into the network server with what you want to have happen to the decrypted application level traffic coming out of the network server.

So the issue now is, I dont get to see any traffic/data that I have subscribed to.

Indeed, you won’t, because you broke the MQTT bridge connection between the gateway component and the the network server component of the box.

Reset to factory configuration and configure the network server’s output MQTT; don’t mess with the configuration of the MQTT bridge as it has nothing to do with your goal. These are entirely distinct uses of MQTT.

And remember of course that if you are using the internal server, then you are not using TTN. And if you are using TTN, then you are not using the internal server. OTAA nodes will have to be forced to re-join if you change network servers between internal and TTN.

#24

Thank you for your help/advice. Much appreciated.

I set up the RAK7249 LoRaPacket Server = built-in LoRa server. Left the MQTT bridge alone. Set up Au915 and Concentrator 1. All good. Can see traffic on the LoRaWAN Packet Logger. Pretty cool.

LoRa Network Server Gateway overview = setup Gateway and Applications.

Installed MQTT.fx on a windows 10 laptop

In MQTT.fx topic = application/{{application_ID}}/device/{{device_EUI}}/rx

Pretty soon discovered that “application_ID” is the ID from the :“LoRa Network Server Application Overview” screen and not the ID from the “TTN application/device screen”. Big learning.

Worked perfectly. Got packets on the laptop screen. Doubly pleased and grateful for all the help.

Next issue was to set an MQTT Broker up on an Arduion Uno. I used the following sketch -

#include <SPI.h>
#include <Ethernet.h>
#include <PubSubClient.h>

byte mac[] = {0xDE, 0xED, 0xBA, 0xFE, 0xFE, 0xED}; // Update these with values suitable for your network.
IPAddress ip(192, 168, 1, 50); // IP address to use if DHCP failed
IPAddress server(192, 168, 1, 146); // IP address of RAK7294 built-in server
EthernetClient ethClient;
PubSubClient client(ethClient);

void setup()
{
Serial.begin(57600);
client.setServer(server, 1883);
client.setCallback(callback);
Ethernet.begin(mac, ip); // try to conigfure using IP address instead of DHCP:
delay(5000); // Allow the hardware to sort itself out
}

void loop()
{
if (!client.connected())
{
reconnect();
}
}

void reconnect()
{
while (!client.connected()) // Loop until we’re reconnected
{
Serial.print(“Attempting MQTT connection… \n”);
if (client.connect(“arduinoClient”)) // Attempt to connect
{
Serial.println(“connected”);
client.subscribe(“application/3/device/00033bbcff12a631/rx”); // Once connected,resubscribe
if (client.subscribe(“application/3/device/00033bbcff12a631/rx”)) Serial.print(“sending the subscribe succeeded”);
}
else
{
if (client.state() == -4) Serial.print(“The server didn’t respond within the keep=alive time.”);
if (client.state() == -3) Serial.print(“The network connection was broken.”);
if (client.state() == -2) Serial.print(“The network connection failed.”);
if (client.state() == -1) Serial.print(“The client is disconnected cleanly.”);
if (client.state() == 1) Serial.print(“The server doesn’t support the requested version of MQTT.”);
if (client.state() == 2) Serial.print(“The server rejected the client identifier.”);
if (client.state() == 3) Serial.print(“The server was unable to accept the connection.”);
if (client.state() == 4) Serial.print(“The username/password was rejected.”);
if (client.state() == 5) Serial.print(“The client was not authorised to connect.”);
Serial.println("\n try again in 5 seconds");
delay(5000); // Wait 5 seconds before retrying
}
}
}

void callback(char* topic, byte* payload, unsigned int length)
{
Serial.print(“Message arrived [”);
Serial.print(topic);
Serial.print("] ");
for (int i = 0; i < length; i++)
{
Serial.print((char)payload[i]);
}
Serial.println();
}

I have put in some serial.prints to debug.

The monitor reads -

Attempting MQTT connection
connected
sending the subscribe succeeded

After that, nothing happens. The topic = “application/3/device/00033bbcff12a631/rx” on the MQTT.fx gets a great response of messages: temperature, etc, but on the Arduino, nothing.

Can you please give some ideas as to where Im going wrong.

Again, thank you for your earlier assistance. The outcome has been pretty fantastic. Regards

#25

No, that’s an MQTT client.

As far as I can tell, you are using the MQTT broker built into the gateway.

You then installed MQTT client software on the windows machine, and were able to subscribe to the application feed, which indicates everything is working correctly.

You then failed to get any messages when attempting to subscribe from an Arduino sketch, while the windows setup confirms they are continuing to be published. That would indicate either a problem with the Arduino sketch, or a problem with whatever it is using for networking. If I had to guess, there’s probably an MQTT method you should be calling periodically in your loop() which you are forgetting to call. Events will happen as callbacks, but you need to let the MQTT code run regularly to maintain the connection and find out if there are any messages to deliver to your callback. But that’s really about the Arduino library, not the gateway…

#26

Hi Chris. Thank you for your clarification on nomenclature and your potential source of the problem. That helped immensely.

The MQTT client hardware is an Arduino Uno with a “Mega compatible” Ethernet Shield. It does not have a MAC address label on it, so I assigned one in the sketch. I also set up a static a IP address via the DHCP in the router using the assigned MAC address. I also removed the “Client ID” from the LoRa Gateway MQTT Bridge in the RAK7249 because the Arduino sketch sets that up. I loaded the code. I also used “Advanced IP Scanner” software to see if the IP/MAC address was active. It was.

The Arduino MQTT Client connected, advised “message received”, and printed out the payload on the laptop monitor. Huge success. Thank you again.

I have two LoRa test nodes operating to the 7249 gateway. One says “Hello, Ryan” (after my grandson). The other sends DHT22 sensor information (temperature and humidity) to the gateway. Their topic names and payload sizes are as follows -

DHT22 “application/2/device/01f3b8fb5a1c1050/rx” payload size 4
Hello Ryan “application/3/device/00033bbcff12a631/rx” payload size 12

The DHT22 works perfectly to the Arduino MQTT client, however, when I substitute the topic in the sketch and re-compile, load, etc, I get a message to say “The client is connected” and then it sits forever waiting for the message on the subscribed topic.

So, put simply, the set up doesn’t work following a simple change of topic.

I have re-booted the router, rebooted the RAK7249, changed Arduinos, changed Ethernet shields to the genuine Arduino version, double checked the 7249 settings, checked that messages are indeed being sent on the new topic by monitoring message behaviour on both topics via MQTT.fx running on my laptop; the messages are there. Checked the IP Scanner for the connection to the network.

Wondered if you have any ideas on this please. Thank you. Regards

#27

You shouldn’t have touched the LoRa Gateway Bridge settings at all. That’s what I’ve been explaining over and over - that MQTT communication is internal to the gateway alone. The gateway bridge carries only raw traffic, it does not carry application topics. And even if this were a mis-statement, an MQTT client ID describes a client, not the traffic. Each client has its own ID, fixed or dynamically assigned. It’s not usually very relevant except in MQTT broker logs.

No particular idea what’s going on with the Arduino, but very concerned about apparent misconfiguration of the gateway.

One thing you can do with an ordinary MQTT client is subscribe to wildcard topics, I don’t know if your Arduino library supports that or not. It’s also possible that a message is being received, but your parsing code is getting (permanently?) stuck on the way to trying to print it. Generally you shouldn’t send ASCII strings over LoRaWAN as they are horribly inefficient.

I’m not familiar with your PC MQTT client but with something like mosquitto_pub you could also easily re-inject already received traffic by publishing it again with slight variations, or mock traffic entirely. That might let you iterate and figure out what exactly is causing difficulties for the Arduino.

#28

Hi Chris. Leave the LoRa Gateway bridge settings alone: I get that. I was indeed returning the setting to its original. I had put the ClientID in there some time ago, so I too it out.

I have used an external site that allows you to publish and subscribe and that worked well.

I’m sending HEX coded payloads.

The thing is that the parsing code works of with one of the above topic but the same sketch doesn’t work with the other one.

Secondly, both topics (above) with heaps of traffic are visible on MQTT.fx client.

I think I’ll start aver. Factory reset of the 7249 and go from there.

Thanks for your insights. Regards

#29

I’d try publishing mock messages back into the application broker from the PC, and see what the Arduino gets.

Eg, you could take the message that works and change it bit by bit towards the one that doesn’t. Does it work if you keep the topic the same but change the contents? Or the reverse?

#30

Hi Chris. Thank you for your response. I guess the “verbose” version of what I did is pretty much along the lines that you recommend. I started by setting up an Ethernet server, then an Ethernet client to make sure that the IP and MAC addressing was working for the Arduino. Then I set up an Arduino MQTT client that published to a web site, then subscribed to that site and read the message back. Factory reset the 7249 to ensure the MQTT was pristine, then chose the built in LoRa Server. I watched the packet logger on the 7249, and also set up the MQTT.fx to make sure the build-in server was doing its thing, and then saw the payloads appear on the MQTT.fx. Also looked on the IP Scanner to see that the IP address was operating.

I put this topic into the Arduino Client -

“application/2/device/01f3b8fb5a1c1050/rx”

It took a few seconds but I got messages. The payload was printed on the monitor. In my view it worked perfectly, and remained working like that for a day or so.

Then I changed the topic to -

“application/3/device/00033bbcff12a631/rx”

I got a message -

“Attempting the MQTT connection …”

Followed by -

“The client is connected”.

I got nothing after that. So an absolutely working MQTT client stopped working after changing the client from -

“application/2/device/01f3b8fb5a1c1050/rx”

to -

“application/3/device/00033bbcff12a631/rx”.

So I feel that I have already done the this that you suggest.

However, starting from scratch might be useful.

Thank you for your continued assistance. Regards.

#31

That’s a change of topic. The clients are the RAK7258’s network server which is publishing messages, and your PC software and Arduino software which are subscribed to them.

I feel that I have already done the this that you suggest.

What I suggested what that you use your PC MQTT software to publish mock messages back into the MQTT broker, and see if the Arduino gets them.

You could first try to exactly repeate the working messages. Then you could do things like send the payload of the non-working message but to the topic of the working one. Or the other way around.

And I also suggested you look into seeing if the Arduino MQTT software supports wildcards in the topic subscriptions the way standard MQTT client software does. So see if you can subscribe to “application/*”

#32

Hi Chris - Thank you again, especially for your patience.

With regard to what you suggested re wildcard in the topic. It appears that a “+” works. On a “*” or a “#” the monitor goes round and round at hi speed telling me I’m attempting to connect. I’m connected. I’m attempting to connect. I’m connected … However with a “+” the topic becomes -

mqttClient.subscribe(“application/+/device/+/rx”);

and I get the following screen shot -

image

You will see that I never go to any other application, eg application 3, in the subscribed topic

Thanks. Regards

#33

Hi Chris. Progress I think.

I changed the topic to -

mqttClient.subscribe(“application/+/device/+/+”);

ie, the uplink, down link, join etc to a wildcard, and then I restarted the Arduino node and here is the screenshot -

image
image1039×377 50.3 KB

You will notice that the second-to-bottom message is from “application 3” and you will see the topic ends with “join”.

So it appears that that node is attempting to join but there is no subsequent uplink.

How di I fix that, please?

#34

You’d have to figure out why it is failing.

Are join accepts being generated, eg in the gateway screen do you see it transmitting replies?

Often this is a timing problem on the node, but it can be a configuration problem, etc. Really debugging it can require digging deep into node firmware.

#35

Hi Chris. I cld not work out why messages from that particular node cld not be seen. Did a lot of research and finally decided maybe I’ll set up a brand new node. different code, different message. I did that and the Arduino MQTT Client now works perfectly. Very happy about that. Many thanks for all your help. Best wishes for 2021. Kind regards.

#36

Hello, again. Same topic, slightly different problem.

I am using RAK7249/4240 with in-build LoRa server. I have built an Arduino MQTT client, and connected it to the 7249. I have a couple of nodes working, one sending DHT22 temp/humidity information, the other sending “Hello, Ryan”. The Arduino MQTT client is subscribed to messages from these two nodes. I have managed to get this configuration working perfectly. I can re-set the nodes individually, see the “join” messages and the “Rx” messages thereafter on both the Arduino MQTT and the Laptop MQTT.fx. And I can see the payloads. Very happy.

I also have MQTT.fx running on a laptop, also connected to the 7249 and I can see the same messages/payloads on the laptop screen. So all good.

On the weekend I upgraded to RAK7249 firmware to the latest version. I must also point out that while trying to remove the ethernet RJ45 plug a couple of weeks ago, the RJ45 pcb-mounted socket came off in my hand. While I await a replacement, I tried to connect via RAK7249 access point. This has been the subject of a separate thread in that I could not get the 7249 to establish a client relationship with my Linksys LAPAC access point so that I could keep my applications running. Thanks to Todor’s very generous assistance in a middle-of-the-night intervention, I now have that client relationship working fine

The problem now is that while I can see the “join” requests and the “Rx” responses and payloads from my 2 nodes on the laptop MQTT.fx, I can only see the “join” request and not the “Rx” responses on my Arduino MQTT client nor the payloads.

There has been no change to the Arduino hardware, The IP address of the server is the only software change, and I am messaged that the Arduino has connected, and I do get the “join” message.

The only real change is the firmware upgrade to the 7249.

Any suggestions, please.

#37

Use your PC-based subscription to figure out what’s different about the feed, particularly between what you receive there and not at the Arduino, and adjust your Arduino sketch accordingly.

Look at things like topic, encoding (formats or funny characters) and even overall length