Aha. You are right! I have not got your point. 
Much appreciated. I will fix it asap.
1 Like
@jpmeijers I capture the UDP data via Wireshark. It seems that the json was not truncated.
The max UDP payload size defined in packet forwarder is 4640 byte. (max 8 uplink lora packet). The sizeof the captured pkt is 4424 which is less then the max size. But the TTN not send ACK for it.
Maybe the Max UDP payload size defined in TTN is less then 4420.
Tcpdump in gateway cannot print too long message. So it looks to be truncated. But in fact not.
Ahh ok, interesting result.
Could it be that we are hitting the MTU size of the network, and that fragmentation is causing an issue?
Perhaps it is because the network delay is too large to tolerate fragmentation. Or the UDP payload size defined by TTN is too small.
When I reduce the UDP payload size (less then the MTU), it works fine.
I’m reading up on this and some people say it is going to be an issue, as UDP does not re-assemble and re-order the fragments like TCP does:
https://info.stack8.com/blog/udp-fragmentation-why-should-you-avoid-it
Specifically:
However, on the other hand, UDP being a message oriented protocol, it does not have a built-in reordering or retransmitting mechanism, so fragmentation should be avoided. Further, when your traffic flows through devices that you have no control over nor visibility on such as sending traffic over the internet, then this should be avoided at all cost.
Maybe we should be safe and only send a single uplink message per UDP packet? And not put more than one inside a UDP packet. We have no idea what the MTUs are of all the hops of the network. It could be very small if some VPNs are used inside eachother. Best is to remain on the safe side and stay far below the standard 1500byte MTU.
Emm. Two uplink message per UDP packet is also safe. The max sizeof per uplink message is 540. But this will cause a decrease in efficiency. Not every message has 540 bytes. Most messages are only a few dozen bytes.
I will try to dynamically adjust the number of messages in each UDP packet to ensure that the total length is less than the MTU.
1 Like
Perfect!
Path MTU discovery in practice says the following:
The minimal required MTU for all IPv6 hosts is 1,280, which is fair. Unfortunately for IPv4 the value is 576 bytes. On the other hand RFC4821 suggests that it’s “probably safe enough” to assume minimal MTU of 1,024.
So maybe use 1024?
Important thing to keep in mind is that the MTU you see on the gateway is not the minimum MTU of all the links on the path taken by the IP packets. It’s only the MTU for the first hop from the gateway to the first router.
Another problem is that the packet forwarder only send the next UDP packet after receive the ACK of the last one. If loragateway receives multiple long messages at once, it must be sent multiple times. If the network delay is large, the packets sent later may miss Rx1Window.
Good one! Definitely one of the issues with using the UDP packet forwarder. Basic Station should solve this.
But at this point Data Recovery is anyway very old data that missed the RX windows long time ago. So it’s fine to go on as we are.
I will try to send all udp pkt before collect the ACK.
1 Like
Hi @yutao. Are you succeeding to make the changes to the packet forwarder? Let me know if you have anything I can test.
This was solved in V1.1.0063_Release_r205
I trying to write logic on my application side to handle cases like these. For RAK gateways I trust the gateway’s timestamp, as the packets can be buffered and sent to the network at quite some time later. Therefore the network time is incorrect.
However, the gateway metadata does not contain a time:
{
"app_id":"myapp",
"dev_id":"mydevice",
"hardware_serial":"1122334455667788",
"port":130,
"counter":2388,
"payload_raw":"JNAITQhmCIsI",
"metadata":{
"time":"2021-04-21T13:55:19.427237979Z",
"frequency":868.1,
"modulation":"LORA",
"data_rate":"SF11BW125",
"airtime":741376000,
"coding_rate":"4/5",
"gateways":[
{
"gtw_id":"eui-ac1f09fffe013d9b",
"timestamp":1442497396,
"time":"", // <== THIS IS EMPTY!!
"channel":0,
"rssi":-124,
"snr":-12.8,
"rf_chain":0
}
]
}
}
Because the time field for the gateway is empty, I do not know when the packets were received, and therefore I can not use the automatic data recovery feature to back fill missed data.
Is there a way we can have the time field populated so that the data recovery feature can be used?
You got this message from TTN, right ? Why not use timestamp as the time ? I guess the time field was for the gateway who has GPS inside.
Please see packet_forwarder/PROTOCOL.TXT at master · Lora-net/packet_forwarder · GitHub part 4.
-
timeis the wall time as an iso string. -
tmmsis the GPS timestamp -
tmstis an internal counter keeping track of when packets are received and when downlinks should be scheduled. All relative time, and has no connection to wall time. You can see this when trying to interpret1442497396above as a unix timestamp.
Two more examples. You will see how the timestamp value increased a lot more than what a unix timestamp would have.
{"app_id":"myapp","dev_id":"mydevice","hardware_serial":"redacted","port":130,"counter":2627,"payload_raw":"JDoHAghyCJEI","metadata":{"time":"2021-04-22T06:56:55.901018097Z","frequency":867.7,"modulation":"LORA","data_rate":"SF11BW125","airtime":741376000,"coding_rate":"4/5","gateways":[{"gtw_id":"eui-ac1f09fffe013d9b","timestamp":2609434412,"time":"","channel":6,"rssi":-116,"snr":-7,"rf_chain":0}]}}
{"app_id":"myapp","dev_id":"mydevice","hardware_serial":"redacted","port":31,"counter":2628,"payload_raw":"AAA=","metadata":{"time":"2021-04-22T06:58:01.280731299Z","frequency":868.3,"modulation":"LORA","data_rate":"SF11BW125","airtime":659456000,"coding_rate":"4/5","gateways":[{"gtw_id":"eui-ac1f09fffe013d9b","timestamp":2674798644,"time":"","channel":1,"rssi":-120,"snr":-9.5,"rf_chain":0}]}}
Timestamp is microseconds, gateway unique, rolls over about hourly and resets any time the concentrator chip does, so you can’t really use that.
Realistically packet archiving is incompatible with the LoRaWan spec prohibition of out of sequence frame counts, especially in a public network with gateways run by others. If you want to do that you probably need custom software to send the archived packets somewhere other than TTN for independent validation and decryption.
Or you can guess at archived packet times based on frame count and assuming a somewhat consistent interval between packet attempts.
Indeed, but in some situations there is no other way to prevent data loss during power outages.
A couple of gateway manufacturers implemented this feature (Kerlink).
Using Lacuna’s satellite system, which stores and forwards data to a LoRaWAN network, the same issue will happen. Therefore The Things Stack V3 will be adding a special channel via which buffered packets can be uploaded, without affecting the mac state.
See Support downstream resending traffic out-of-order · Issue #2708 · TheThingsNetwork/lorawan-stack · GitHub
Regarding the RAK gateway, it’s a workaround that is needed until TTS implements the new api.
@yutao can we get the gateway’s time added into the time field? If the gateway is NTP synced, this time is accurate enough.