Postgresql db not starting/responding chirpstack

Issue: storage: ping PostgreSQL database error. DB not available or seen by app server.

Setup: RAK7244. rpi4 latest RAK software flashed and installed on 64GB card.

Server: chirpstack latest stable. everything on localhost

Details: I downloaded and installed latest RAK7244 image and selected chirpstack and US915. I was actually able to get the gui on localhost:8080 and setup one device (LHT65).
Not sure what happened or what I did but I lost the gui and it seemed that the db was no linger responding.

Jul 20 15:10:59 rak-gateway chirpstack-network-server[611]: time=“2021-07-20T15:10:59.061999361-04:00” level=warning msg=“storage: ping PostgreSQL database error, will retry in 2s” error=“dial tcp [::1]:5432: connect: connection refused”

DB gave the following: [unknown]@[unknown] LOG: incomplete startup packet. status indicated that the app server started and then exited because of unknown packet as above.

Thought I had this thing beat but now I’m stuck again. Thanks in advance for any badly needed help.

I tried reflashing another card but got the same errors, although I never got to the gui again on the second try.
BG

I’d do a apt update + apt upgrade to ensure all the components are up to date as a first thing.

But the card may have a glitch in it that has caused a corruption in the database.

Nick,
I thought you had saved me the other day but somehow I messed up. All packages are up to date.
I got some error messages about en_GB.UTF-8 which I changed in the .conf file. This had been set correctly in the raspi-config.
Next error, see below, is my LC_COLLATE "en_US.UTF-8 is in the wrong place in the .conf file. Not sure where it goes but not where I put it.


[email protected]:/home/pi# systemctl status [email protected][email protected] - PostgreSQL Cluster 11-main
   Loaded: loaded (/lib/systemd/system/[email protected]; enabled-runtime; vendor preset: enabled)
   Active: failed (Result: protocol) since Tue 2021-07-20 17:44:09 EDT; 38s ago
  Process: 576 ExecStart=/usr/bin/pg_ctlcluster --skip-systemctl-redirect 11-main start (code=exited, status=1/FAILURE)

Jul 20 17:44:08 rak-gateway systemd[1]: Starting PostgreSQL Cluster 11-main...
Jul 20 17:44:09 rak-gateway [email protected][576]: Error: invalid line 40 in /etc/postgresql/11/main/postgresql.conf: LC_COLLATE "en_US.UTF-8"
Jul 20 17:44:09 rak-gateway systemd[1]: [email protected]: Can't open PID file /run/postgresql/11-main.pid (yet?) after start: No such file or direct
Jul 20 17:44:09 rak-gateway systemd[1]: [email protected]: Failed with result 'protocol'.
Jul 20 17:44:09 rak-gateway systemd[1]: Failed to start PostgreSQL Cluster 11-main.

Thanks if you can figure out what I’ve done.
Cheers,
BG

Should I do a raspi-config before gateway-config? Maybe I messed up here as I did the opposite.

I wouldn’t expect the raspi-config would need to be done as the gateway-config does all the setting up - albeit occasionally not creating a DNS entry.

And in theory, as the image will have been tested before being uploaded :wink:, it should all work without doing any updates.

Maybe trying the previous firmware version?

I’ll try the software straight without messing with raspi-config, only using gateway-config. Maybe the gateway-config takes care of all the localizations, etc. If that doesn’t work then I’ll try an earlier version. Will keep you posted.
Cheers,
[email protected]

Nick,

I’ve made some progress but still getting some errors in syslog. The key to installing this is first install rpi-lite and run raspi-config for localizations, etc. Then “git” the RAK software from their site and run gateway-config for the US915 setup. I do have a desktop on localhost:8080 but it’s showing no activity although I’ve tried to set up the gateway and LHT65 device. I’ve posted some of the syslog and postgresql.log. If you could point out shere I need to start looking for problems then that would be great. Thanks in advance. Let me know if I need to send you more info.

Jul 21 16:45:03 rak-gateway chirpstack-gateway-bridge[636]: time="2021-07-21T16:45:03.253491162-04:00" level=info msg="integration/mqtt: publishing event" event=stats qos=0 stats_id=45c2fb67-a3bc-4d98-82c4-e3c11e8fccea topic=gateway/dca632fffe36b226/event/stats
Jul 21 16:45:03 rak-gateway chirpstack-network-server[635]: time="2021-07-21T16:45:03.255410413-04:00" level=info msg="gateway/mqtt: gateway stats packet received" gateway_id=dca632fffe36b226 stats_id=45c2fb67-a3bc-4d98-82c4-e3c11e8fccea
Jul 21 16:45:03 rak-gateway chirpstack-network-server[635]: time="2021-07-21T16:45:03.260432003-04:00" level=error msg="gateway: handle gateway stats error" ctx_id=45c2fb67-a3bc-4d98-82c4-e3c11e8fccea error="update gateway state error: object does not exist"


[email protected]:/var/log/postgresql $ tail -f postgresql-11-main.log
2021-07-21 16:43:09.427 EDT [750] LOG:  aborting any active transactions
2021-07-21 16:43:09.443 EDT [750] LOG:  background worker "logical replication launcher" (PID 757) exited with exit code 1
2021-07-21 16:43:09.456 EDT [752] LOG:  shutting down
2021-07-21 16:43:09.532 EDT [750] LOG:  database system is shut down
2021-07-21 16:43:43.295 EDT [737] LOG:  listening on IPv6 address "::1", port 5432
2021-07-21 16:43:43.296 EDT [737] LOG:  listening on IPv4 address "127.0.0.1", port 5432
2021-07-21 16:43:43.323 EDT [737] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2021-07-21 16:43:43.511 EDT [738] LOG:  database system was shut down at 2021-07-21 16:43:09 EDT
2021-07-21 16:43:43.572 EDT [737] LOG:  database system is ready to accept connections
2021-07-21 16:43:44.121 EDT [745] [unknown]@[unknown] LOG:  incomplete startup packet

```type or paste code here
Cheers,
BG

I’ll go and “break” one of my gateways to see if I can replicate this - normally things behave when I’m around!

Nick,
After about 10 re-flashes and restarts I decided to try the RAK-latest-firmware again. This time, doing everything from the gateway-config except localizations, I was able to get the gateway and chirpstack gui up and running. The gateway is showing up as active and on the map. My problem now is that I can’t get the LHT65 to join. It’s sending out join requests but the gateway doesn’t reply or acknowledge. I’m attaching some data, both about the RAL7244 and LHT65 along with a couple of live frames seen on the gateway. Maybe your trained eye can see a problem here. Thanks in advance. BG

LHT65 info:
AT+DEUI=a8 40 41 17 b1 83 12 e9  
AT+DADDR=018312E9
AT+APPKEY=cc ce 37 16 12 fb 7c 1c b3 88 db 9a b8 12 33 a2
AT+NWKSKEY=19 8f bf d6 22 a3 7f e5 eb 29 79 81 cd e6 fe 88
AT+APPSKEY=21 a8 72 da d2 d3 8b 3e 64 3d 74 d2 e5 e3 46 28
AT+APPEUI=a0 00 00 00 00 00 01 00
AT+ADR=1
AT+TXP=5
AT+DR=1
AT+DCS=0
AT+PNM=1
AT+RX2FQ=923300000
*****************************
Gateway details
Gateway ID	dca632fffe36b226
Altitude	80 meters
GPS coordinates	37.75323, -77.4851
Last seen at	Jul 23, 2021 9:30 AM
***************************
ChirpStack.io
rak-gateway
gatewayID:"dca632fffe36b226"
frequency:923300000 **** [could this be a problem here?]
power:20
modulation:"LORA"
bandwidth:500
spreadingFactor:8
codeRate:"4/5"
polarizationInversion:false
board:0
antenna:0
timing:"IMMEDIATELY"
immediatelyTimingInfo:

            context:null

mType:"Proprietary"
major:"LoRaWANR1"

            bytes:""
            mic:"e2a46003"

gatewayID:"dca632fffe36b226"
time:"2021-07-23T11:53:40.716049Z"
timeSinceGPSEpoch:"1311076439.716s"
rssi:-53
loRaSNR:11.5
channel:1
rfChain:0
board:0
antenna:0
        latitude:37.75327
        longitude:-77.48488
        altitude:71
        source:"UNKNOWN"
        accuracy:0
        fineTimestampType:"NONE"
        context:"BtTktA=="
        uplinkID:"2ae31cb8-bbe8-4a0d-977d-2f08ce2118cb"
        crcStatus:"CRC_OK"

frequency:904100000 ****[not same as gateway above..?]
modulation:"LORA"
            bandwidth:125
            spreadingFactor:10
            codeRate:"4/5"
            polarizationInversion:false

mType:"JoinRequest"
major:"LoRaWANR1"

            joinEUI:"a000000000000100"
            devEUI:"a8404117b18312e9"
            devNonce:32154
            mic:"df481537"

gatewayID:"dca632fffe36b226"

frequency:923300000
power:20
modulation:"LORA"
bandwidth:500
spreadingFactor:8
codeRate:"4/5"
polarizationInversion:false
board:0
antenna:0
timing:"IMMEDIATELY"
immediatelyTimingInfo:

            context:null

mType:"Proprietary"
major:"LoRaWANR1"

            bytes:""
            mic:"3696bf1d"


On the LHT65, what does the command AT+VER=? tell you?
I looked into the LHT65 AT command manual, but I didn’t see any command to select the region.

Looking into Dragino documentation, it looks like the region is fixed for the device you bought:

LHT65-XXXXX-YY

XXXXX: the default frequency band

    EU433: Default frequency band EU433
    CN470: Default frequency band CN470
    EU868: Default frequency band EU868
    IN865: Default frequency band IN865
    KR920: Default frequency band KR920
    AS923: Default frequency band AS923
    AU915: Default frequency band AU915
    US915: Default frequency band US915

Beegee,
I’m sure the frequency is ok as I had everything setup and working on ttnv2 & ttnv3. The lht65 is making join requests to the RAK7244 but not being accepted. Got an error of incorrect “mic” at one point. I actually had this working on chirpstack once and then did something that messed up and I haven’t been able to return to that point. Got to be something simple that I did wrong.
When setting up the device must be the problem. Thanks.
bg

Nick,
Just wanted to say that I seem to have everything working and am backing up after each step. I’m not sure why I was losing the gui when the postgresql db seemed to disappear but that seems to be working now.

Once I got the gui working it was a question of putting the correct eui and keys where they needed to be. It was rather hit and miss as I wasn’t clear on which to use. The update/upgrade didn’t seem to hurt anything
(so far).

My next project is to attempt to understand MQTT enough to send my LHT65 data to another server, probably emoncms if you are familiar with that software. You will surely see me post again :slightly_smiling_face:

Many thanks and appreciation to you and those who helped.
BG

1 Like