UART1 hang issue with GNSS module

Hello,

I’m experiencing an issue with UART communication in a tracker application using the Rak3172 module and Quectel L76 GNSS module, developed with RUI in the Arduino environment.

Problem Description:

  • The UART1 hangs sporadically after 5-20 minutes of operation.
  • Only the first character from the serial input is read (‘$’), and all subsequent characters are lost.
  • This issue occurs more frequently when the GNSS module sends more messages.
  • The GNSS module is connected to UART1.
  • Changing the UART1 baud rate does not affect the behavior.

During the faulty operation, I used a logic analyzer to check the UART line and confirmed that the NMEA sentences were presented correctly. However, the Rak3172 only received the first character, suggesting a possible UART buffer overflow. This could be due to high interrupt load preventing the DMA buffer content from being copied to the UART RX buffer. I have not confirmed this with a debugger.

The issue is present in the following RUI versions: 3.5.6, 4.0.0, and 4.1.0 (tested only with these versions).

Workaround:
I modified the UART buffering in the BSP to use circular DMA mode. This change has resolved the issue, and I have not experienced the faulty behavior since. However, this workaround has a drawback: I can no longer reconfigure the UART baud rate after initialization.

The issue can be reproduced with the following application code

#include <TinyGPSPlus.h>

#define LogSerial Serial
#define GpsSerial Serial1

// LoRA parameters
#define CHANNEL_NUM         0

#define LORA_SF_RX           9
#define LORA_SF_TX           11
#define LORA_BW              1
#define LORA_CR              0
#define LORA_PREAMBLE        8
#define LORA_TX_POWER        14

#define SEND_PERIOD        45000
uint32_t channels[] = {868200000};

uint32_t myFreq;            // Hz
uint16_t sf;                // [SF7..SF12]
uint16_t bw;                // kHz 0 = 125, 1 = 250, 2 = 500, 3 = 7.8, 4 = 10.4, 5 = 15.63, 6 = 20.83, 7 = 31.25, 8 = 41.67, 9 = 62.5
uint16_t cr;                // 4/(cr+5)
uint16_t preamble;          // Same for Tx and Rx
uint16_t txPower;           // dBm

// The TinyGPSPlus object
TinyGPSPlus gps;


void recv_cb(rui_lora_p2p_recv_t data)
{
  LogSerial.printf("Incoming message, length: %d, RSSI: %d, SNR: %d", data.BufferSize, data.Rssi, data.Snr);
}

void send_cb(void)
{
    // After Tx set back to Rx mode   
    set_radio_sf_rx(); 
    bool is_in_rx{false};
    do{    
      is_in_rx = api.lorawan.precv(65534);
    }
    while(!is_in_rx);
}

void send_routine_periodic()
{
  uint8_t payload[25] = {0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4}; //dummy message
  
  set_radio_sf_tx();
  bool send_result = false;
  while (!send_result) {
    send_result = api.lorawan.psend(sizeof(payload), payload);
    if (!send_result) {
      api.lorawan.precv(0);
    }
  }
}

void routine_100ms()
{
  listen_gps();
}

void listen_gps()
{
  uint16_t cntr = 0U;
  char c;

  while(GpsSerial.available())
  {
    cntr++;
    c = GpsSerial.read();
    gps.encode(c);
  }
  LogSerial.printf("%d\r\n",cntr); //Logging number of received characters
}


void set_radio_sf_rx()
{
  sf = LORA_SF_RX;
  api.lorawan.psf.set(sf);
}

void set_radio_sf_tx()
{
  sf = LORA_SF_TX;
  api.lorawan.psf.set(sf);
}

void set_radio_rx_init()
{
  myFreq = channels[CHANNEL_NUM];
  sf = LORA_SF_RX;
  bw = LORA_BW;
  cr = LORA_CR;
  preamble = LORA_PREAMBLE;
  txPower = LORA_TX_POWER;
  radio_init();
  LogSerial.println(F("RX mode"));
}

void radio_init()
{
  LogSerial.printf("Set P2P mode frequency %3.3f: %s\r\n", (myFreq / 1e6),
  api.lorawan.pfreq.set(myFreq) ? "Success" : "Fail");
  LogSerial.printf("Set P2P mode spreading factor %d: %s\r\n", sf,
  api.lorawan.psf.set(sf) ? "Success" : "Fail"); //P2P Spreading Factor (6, 7, 8, 9, 10, 11, 12)
  LogSerial.printf("Set P2P mode bandwidth %d: %s\r\n", bw,
  api.lorawan.pbw.set(bw) ? "Success" : "Fail"); //P2P bandwidth in kHz (0 = 125, 1 = 250, 2 = 500, 3 = 7.8, 4 = 10.4, 5 = 15.63, 6 = 20.83, 7 = 31.25, 8 = 41.67, 9 = 62.5)
  LogSerial.printf("Set P2P mode code rate 4/%d: %s\r\n", (cr + 5),
  api.lorawan.pcr.set(cr) ? "Success" : "Fail");
  LogSerial.printf("Set P2P mode preamble length %d: %s\r\n", preamble,
  api.lorawan.ppl.set(preamble) ? "Success" : "Fail"); //P2P Preamble Length (2-65535)
  LogSerial.printf("Set P2P mode tx power %d: %s\r\n", txPower,
  api.lorawan.ptp.set(txPower) ? "Success" : "Fail"); //P2P TX Power (5-22)
}

void setup() {
  LogSerial.begin(256000);
  
  if(api.lorawan.nwm.get() != 0)
    {
        LogSerial.printf("Set Node device work mode %s\r\n",
            api.lorawan.nwm.set(0) ? "Success" : "Fail"); // network working mode (0 = P2P, 1 = LoRaWAN, 2 = FSK)
        api.system.reboot();
    }
  
  set_radio_rx_init();
    api.lorawan.registerPRecvCallback(recv_cb);
    api.lorawan.registerPSendCallback(send_cb);

    bool is_in_rx = api.lorawan.precv(65534);
    LogSerial.printf("P2P set Rx mode %s\r\n",
      is_in_rx ? "Success" : "Fail");
    
  GpsSerial.begin(9600);
    
  if (api.system.timer.create(RAK_TIMER_0, (RAK_TIMER_HANDLER)send_routine_periodic, RAK_TIMER_PERIODIC) != true) {
    LogSerial.println(F("LoRa - Creating timer failed."));
    }
  
    if (api.system.timer.start(RAK_TIMER_0, SEND_PERIOD, NULL) != true) {
        LogSerial.println(F("LoRa - Starting timer failed."));
    }
    if (api.system.timer.create(RAK_TIMER_1, (RAK_TIMER_HANDLER)routine_100ms, RAK_TIMER_PERIODIC) != true) {
        LogSerial.println(F("GPS - Creating timer failed."));
    }
    if (api.system.timer.start(RAK_TIMER_1, 100, NULL) != true) {
        LogSerial.println(F("GPS - Starting timer failed."));
    }
}

void loop()
{
  api.system.sleep.cpu();
}

Application software operation:

  • The application operates in P2P mode with continuous RX, switching to TX only while transmitting data.
  • Different spreading factors are used for RX and TX, while other LoRa parameters remain the same.
  • A 100ms timer is used to read the UART buffer and parse NMEA sentences.
  • A 45-second timer is used to send a LoRa message.

Has anyone else experienced similar issues?

Thanks for reporting this.
I never tried to use GNSS modules over UART since we have the u-blox ZOE-M8Q modules with I2C communication.

Did you try a Serial2.end() and a new Serial2.begin(9600)` before reading from the UART?

Are you permanently reading from the GNSS module? Is this necessary? As the GNSS modules are quite power hungry, most apps I know keep the GNSS module off unless they really need to get a location.

Hi @beegee,

Thanks for your reply. The example code I attached is just to easily reproduce the issue. It represents more or less the active mode of the GNSS tracker. Of course, in low power mode, the GNSS module is in standby mode. The GNSS module is connected to UART1. Unfortunately, deinitializing and initializing the serial port did not solve the issue.

I had an issue with UART initialisation at the start of last year. (I was talking to RAK12500 over NMEA/UART then but have since moved to UBX/I2C).

The workaround that applied in my case was to initialise the serial port twice. I’ll find the example … here it is

/*

  Demo code to talk to the ZOE M8Q GNSS module on the RAK 12500 via UART and NMEA

*/

void setup()

{

  // Initialize Serial for debug output

  Serial.begin(115200);

  while (!Serial);  // Wait for Serial port to be available, no time out

  delay(100);       // There needs to be some delay here. Not sure how much

  Serial.println("GPS ZOE-M8Q (NMEA over UART)");

  Serial.printf("Firmware Version: %s\r\n", api.system.firmwareVersion.get().c_str());

  delay(1000);    // Window to enter AT+BOOT when it all goes wrong

  // Reset the RAK12500 GNSS sensor, plugged into slot A

  pinMode(WB_IO2, OUTPUT);

  digitalWrite(WB_IO2, 0);

  delay(1000);

  digitalWrite(WB_IO2, 1);

  delay(1000);

  // Initialise internal UART

  Serial1.begin(9600, RAK_CUSTOM_MODE);  // default UART baud for ZOE-M8Q

  while (!Serial1);     // Wait for it to be ready

  Serial1.end();

  // Re-Initialise internal UART

  Serial1.begin(9600, RAK_CUSTOM_MODE);  // default UART baud for ZOE-M8Q

  while (!Serial1);     // Wait for it to be ready

  Serial1.end();

  Serial.println("Starting serial relay");

}

void loop()

{

  // By default the following NMEA strings should be enabled on UART:

  // GGA GLL GSA GSV RMC VTG TXT

  // I can see them on the scope on J10-4

  // Just try echoing

  if( Serial1.available() )

    Serial.print((char) Serial1.read());

}

It made no sense at the time, and it make no more sense now, but that was the trick that worked for me.

I can reproduce your problem and I have no solution beside of that you have to rethink your code.

(1) you are checking Serial1 every 100ms and sending a report over Serial. That is too fast and while sending over Serial, it could still be that the next timer event is already triggered
(2) you continue to check Serial1 every 100ms while you are transmitting data. Keep in mind that the LoRa drivers in the background need some time as well to work properly.
(3) this is not Arduino, RUI3 has a scheduler running in the background for several tasks it has to do. Your code is blocking the scheduler from doing its job.

With some tricks I get your code to work for a longer time, but it is not stable.

Hi!

I have made some code changes:

  • Increased the 100ms task period to 500ms (responsible for parsing the incoming NMEA sentences)
  • Disabled the UART with Serial.end() before LoRa TX and re-enabled it when TX finished
  • Changed LoRa parameters
  • Increased the baud rate of the GNSS module

Those changes had no impact.

I think the issue is not related to the scheduler.

Let’s see how the UART reception is done in the RAK BSP:
Received data is copied to a 128-byte buffer by the DMA. If this DMA buffer is full or UART is in IDLE, the HAL_UART_RxCpltCallback() function is called, which disables the DMA and copies the DMA data to a circular buffer, then re-enables the DMA. If something high-priority happens (a LoRa TX in my case) between DMA disable and enable, it can cause the DMA to be disabled for a longer period. If new characters are received by the UART peripheral while the DMA is disabled, it causes a UART overrun error. I checked the ORE flag of UART1, and it is set when the issue occurs. Most probably, the copy from the DMA buffer to a circular buffer is preempted during LoRa transmission.

The ORE flag can be logged out with:

  volatile uint32_t uart_isr_ore_flag = (((USART1 -> ISR) & USART_ISR_ORE_Msk) >> USART_ISR_ORE_Pos);
  LogSerial.printf("ORE: %d\r\n", uart_isr_ore_flag);

With the updated UART driver, the issue seems to be resolved:
I configured circular DMA mode to UART1, so the DMA can still run while the data is copied from the DMA buffer to the circular buffer.

BR,
András

As a proof-of-concept I put together from other code I have an example for GNSS location acquisition. It works with I2C and Serial communication to an GNSS module on RUI3.

It works fine on a RAK3172, but it will unfortunately not help you much. It uses UBX messages from the GNSS module, which are shorter and easier to decode than the NMEA messages. I testet it on u-blox MAX-7Q (RAK1910, Serial1) and u-blox ZOE-M8Q (RAK12500, I2C),

Unfortunately your module does not support NMEA messages types. It seems to have an I2C port, but that does not offer UBX messages either.

RUI3-Best-Practice ==> RUI3-GNSS

1 Like