[LoRaWAN] Fix channel logic: for fixed bands, persistence of dynamic bands, rejoining during active session

[StevenCellist]

Under various situations the channel logic was not completely tight. Thanks to @jerryneedell for verifying and debugging fixed-band behavior in #935. This PR also fixes bugs on sessions with ADR disabled, as after restore, all channels were masked away. Moreover, rejoining on OTAA during an active session didn't clear the MAC command queue (they piled up) and remained stored channels in the availableChannels structure.

As of 6.4.0, only specific sessions work well (with ADR enabled or without restoring the session, without rejoining during active session). I would suggest a bugfix release to 6.4.1, although that's completely up to you. I have no idea what your policy is around this sort of stuff :)

[StevenCellist]

Apologies for not delivering a bug-free experience on the previous version.. I hope it's much better now!

[StevenCellist]

Alright that were two more.. the setDatarate function accidentally had incorrect equality checks. And with the new key checksum & mode check, I accidentally wiped the DevNonce and JoinNonce as well which should not be discarded if the user just wants to refresh the session keys by doing a new join. That's now fixed and I'm officially out of things I would change for at least the next month unless some user finds a bug.

[StevenCellist]

Nope, still found an off-by-one mistake - your logging struct working wonders in finding these bugs!
Will fix soon when back at my desk.

[jgromes]

@StevenCellist no worries, haven't had the time to review anyway ;)

[jgromes]

@StevenCellist finally had the time to review - there's nothing that would prevent me from merging this, thank you very much for the continued support! I will also run the CI to build everything and 6.4.1 should get released in few hours from now.

[nmaas87]

Short feedback for you (@StevenCellist / @jgromes), I tried a STM32WL55 module with the latest 6.4.1 patch, as well as with 6.4.0 and 6.3.0 in EU868 state with the LoRaWAN reference example.

6.3.0 - works flawlessly
6.4.0 - node.restore() does not work, but initial joining/OTAA does work
6.4.1 - initial joining/OTAA does not work with reference to the region code / status / error code:

[LoRaWAN] Resuming previous session ... failed, code -1101
[LoRaWAN] Attempting over-the-air activation ... failed, code -1106

Am I the issue? :)

/*
  RadioLib LoRaWAN End Device Reference Example

  This example joins a LoRaWAN network and will send
  uplink packets. Before you start, you will have to
  register your device at https://www.thethingsnetwork.org/
  After your device is registered, you can run this example.
  The device will join the network and start uploading data.

  Also, most of the possible and available functions are
  shown here for reference.

  LoRaWAN v1.1 requires the use of EEPROM (persistent storage).
  Please refer to the 'persistent' example once you are familiar
  with LoRaWAN.
  Running this examples REQUIRES you to check "Resets DevNonces"
  on your LoRaWAN dashboard. Refer to the network's 
  documentation on how to do this.

  For default module settings, see the wiki page
  https://github.com/jgromes/RadioLib/wiki/Default-configuration

  For full API reference, see the GitHub Pages
  https://jgromes.github.io/RadioLib/
*/

// include the library
#include <RadioLib.h>

// no need to configure pins, signals are routed to the radio internally
STM32WLx radio = new STM32WLx_Module();

// set RF switch configuration for Nucleo WL55JC1
// NOTE: other boards may be different!
//       Some boards may not have either LP or HP.
//       For those, do not set the LP/HP entry in the table.
static const uint32_t rfswitch_pins[] =
                         {PC3,  PC4,  PC5};
static const Module::RfSwitchMode_t rfswitch_table[] = {
  {STM32WLx::MODE_IDLE,  {LOW,  LOW,  LOW}},
  {STM32WLx::MODE_RX,    {HIGH, HIGH, LOW}},
  {STM32WLx::MODE_TX_LP, {HIGH, HIGH, HIGH}},
  {STM32WLx::MODE_TX_HP, {HIGH, LOW,  HIGH}},
  END_OF_MODE_TABLE,
};

// create the node instance on the EU-868 band
// using the radio module and the encryption key
// make sure you are using the correct band
// based on your geographical location!
LoRaWANNode node(&radio, &EU868);

// for fixed bands with subband selection
// such as US915 and AU915, you must specify
// the subband that matches the Frequency Plan
// that you selected on your LoRaWAN console
/*
  LoRaWANNode node(&radio, &US915, 2);
*/

void setup() {
  Serial.begin(9600);

  // set RF switch control configuration
  // this has to be done prior to calling begin()
  radio.setRfSwitchTable(rfswitch_pins, rfswitch_table);

  // initialize STM32WLx with default settings, except frequency
  Serial.print(F("[STM32WLx] Initializing ... "));
  int state = radio.begin();
  //int state = radio.begin(868.0);
  if(state == RADIOLIB_ERR_NONE) {
    Serial.println(F("success!"));
  } else {
    Serial.print(F("failed, code "));
    Serial.println(state);
    while(true);
  }

  // application identifier - pre-LoRaWAN 1.1.0, this was called appEUI
  // when adding new end device in TTN, you will have to enter this number
  // you can pick any number you want, but it has to be unique
  uint64_t joinEUI = 0x12AD1011B0C0FFEE;


  // device identifier - this number can be anything
  // when adding new end device in TTN, you can generate this number,
  // or you can set any value you want, provided it is also unique
  uint64_t devEUI = 0x70B3D57ED005E120;



  // select some encryption keys which will be used to secure the communication
  // there are two of them - network key and application key
  // because LoRaWAN uses AES-128, the key MUST be 16 bytes (or characters) long

  // network key is the ASCII string "topSecretKey1234"
  uint8_t nwkKey[] = { 0x74, 0x6F, 0x70, 0x53, 0x65, 0x63, 0x72, 0x65,
                       0x74, 0x4B, 0x65, 0x79, 0x31, 0x32, 0x33, 0x34 };

  // application key is the ASCII string "aDifferentKeyABC"
  uint8_t appKey[] = { 0x61, 0x44, 0x69, 0x66, 0x66, 0x65, 0x72, 0x65,
                       0x6E, 0x74, 0x4B, 0x65, 0x79, 0x41, 0x42, 0x43 };

                       

  delay(5000);

  //node.wipe();

  
  // prior to LoRaWAN 1.1.0, only a single "nwkKey" is used
  // when connecting to LoRaWAN 1.0 network, "appKey" will be disregarded
  // and can be set to NULL


  // now we can start the activation
  // this can take up to 10 seconds, and requires a LoRaWAN gateway in range
  // a specific starting-datarate can be selected in dynamic bands (e.g. EU868):
  /* 
    uint8_t joinDr = 4;
    state = node.beginOTAA(joinEUI, devEUI, nwkKey, appKey, joinDr);
  */
  /*
  Serial.print(F("[LoRaWAN] Attempting over-the-air activation ... "));
  state = node.beginOTAA(joinEUI, devEUI, nwkKey, appKey);

  if(state >= RADIOLIB_ERR_NONE) {
    Serial.println(F("success!"));
  } else {
    Serial.print(F("failed, code "));
    Serial.println(state);
    while(true);
  }
  */

  // on EEPROM-enabled boards, after the device has been activated,
  // the session can be restored without rejoining after device power cycle
  // this is intrinsically done when calling `beginOTAA()` with the same keys
  // or if you 'lost' the keys or don't want them included in your sketch
  // you can call `restore()`
  /*
    Serial.print(F("[LoRaWAN] Resuming previous session ... "));
    state = node.restore();
    if(state == RADIOLIB_ERR_NONE) {
      Serial.println(F("success!"));
    } else {
      Serial.print(F("failed, code "));
      Serial.println(state);
      while(true);
    }
  */


    Serial.print(F("[LoRaWAN] Resuming previous session ... "));
    state = node.restore();
    if(state == RADIOLIB_ERR_NONE) {
      Serial.println(F("success!"));
    } else {
      Serial.print(F("failed, code "));
      Serial.println(state);
      //while(true);
  

      node.wipe();
      
      Serial.print(F("[LoRaWAN] Attempting over-the-air activation ... "));
      state = node.beginOTAA(joinEUI, devEUI, nwkKey, appKey);
    
      if(state == RADIOLIB_ERR_NONE) {
        Serial.println(F("success!"));
      } else {
        Serial.print(F("failed, code "));
        Serial.println(state);
        while(true);
      }

       
    }
  

  // disable the ADR algorithm
  //node.setADR(false);

  // set a fixed datarate
  //node.setDatarate(5);
  // in order to save the datarate persistent across reboot/deepsleep, use the following:
  /*
    node.setDatarate(5, true);  
  */

  // enable CSMA
  // this tries to minimize packet loss by searching for a free channel
  // before actually sending an uplink 
  //node.setCSMA(6, 2, true);

  // enable or disable the dutycycle
  // the second argument specific allowed airtime per hour in milliseconds
  // 1250 = TTN FUP (30 seconds / 24 hours)
  // if not called, this corresponds to setDutyCycle(true, 0)
  // setting this to 0 corresponds to the band's maximum allowed dutycycle by law
  node.setDutyCycle(true, 1250);

  // enable or disable the dwell time limits
  // the second argument specific allowed airtime per uplink in milliseconds
  // if not called, this corresponds to setDwellTime(true, 0)
  // setting this to 0 corresponds to the band's maximum allowed dwell time by law
  node.setDwellTime(true, 1000);
}

void loop() {
  int state = RADIOLIB_ERR_NONE;

  // set battery fill level - the LoRaWAN network server
  // may periodically request this information
  // 0 = external power source
  // 1 = lowest (empty battery)
  // 254 = highest (full battery)
  // 255 = unable to measure
  uint8_t battLevel = 146;
  node.setDeviceStatus(battLevel);

  // retrieve the last uplink frame counter
  uint32_t fcntUp = node.getFcntUp();

  Serial.print(F("[LoRaWAN] Sending uplink packet ... "));
  String strUp = "Hello World! #" + String(fcntUp);
  
  // send a confirmed uplink to port 10 every 64th frame
  // and also request the LinkCheck and DeviceTime MAC commands
  if(fcntUp % 64 == 0) {
    state = node.uplink(strUp, 10, true);
    node.sendMacCommandReq(RADIOLIB_LORAWAN_MAC_LINK_CHECK);
    node.sendMacCommandReq(RADIOLIB_LORAWAN_MAC_DEVICE_TIME);
  } else {
    state = node.uplink(strUp, 10);
  }
  if(state == RADIOLIB_ERR_NONE) {
    Serial.println(F("success!"));
  } else {
    Serial.print(F("failed, code "));
    Serial.println(state);
  }

  // after uplink, you can call downlink(),
  // to receive any possible reply from the server
  // this function must be called within a few seconds
  // after uplink to receive the downlink!
  Serial.print(F("[LoRaWAN] Waiting for downlink ... "));
  String strDown;

  // you can also retrieve additional information about 
  // uplink or downlink by passing a reference to
  // LoRaWANEvent_t structure
  LoRaWANEvent_t event;
  state = node.downlink(strDown, &event);
  if(state == RADIOLIB_ERR_NONE) {
    Serial.println(F("success!"));

    // print data of the packet (if there are any)
    Serial.print(F("[LoRaWAN] Data:\t\t"));
    if(strDown.length() > 0) {
      Serial.println(strDown);
    } else {
      Serial.println(F("<MAC commands only>"));
    }

    // print RSSI (Received Signal Strength Indicator)
    Serial.print(F("[LoRaWAN] RSSI:\t\t"));
    Serial.print(radio.getRSSI());
    Serial.println(F(" dBm"));

    // print SNR (Signal-to-Noise Ratio)
    Serial.print(F("[LoRaWAN] SNR:\t\t"));
    Serial.print(radio.getSNR());
    Serial.println(F(" dB"));

    // print frequency error
    Serial.print(F("[LoRaWAN] Frequency error:\t"));
    Serial.print(radio.getFrequencyError());
    Serial.println(F(" Hz"));

    // print extra information about the event
    Serial.println(F("[LoRaWAN] Event information:"));
    Serial.print(F("[LoRaWAN] Direction:\t"));
    if(event.dir == RADIOLIB_LORAWAN_CHANNEL_DIR_UPLINK) {
      Serial.println(F("uplink"));
    } else {
      Serial.println(F("downlink"));
    }
    Serial.print(F("[LoRaWAN] Confirmed:\t"));
    Serial.println(event.confirmed);
    Serial.print(F("[LoRaWAN] Confirming:\t"));
    Serial.println(event.confirming);
    Serial.print(F("[LoRaWAN] Datarate:\t"));
    Serial.print(event.datarate);
    Serial.print(F("[LoRaWAN] Frequency:\t"));
    Serial.print(event.freq, 3);
    Serial.println(F(" MHz"));
    Serial.print(F("[LoRaWAN] Output power:\t"));
    Serial.print(event.power);
    Serial.println(F(" dBm"));
    Serial.print(F("[LoRaWAN] Frame count:\t"));
    Serial.println(event.fcnt);
    Serial.print(F("[LoRaWAN] Port:\t\t"));
    Serial.println(event.port);
    
    Serial.print(radio.getFrequencyError());

    uint8_t margin = 0;
    uint8_t gwCnt = 0;
    if(node.getMacLinkCheckAns(&margin, &gwCnt)) {
      Serial.print(F("[LoRaWAN] LinkCheck margin:\t"));
      Serial.println(margin);
      Serial.print(F("[LoRaWAN] LinkCheck count:\t"));
      Serial.println(gwCnt);
    }

    uint32_t networkTime = 0;
    uint8_t fracSecond = 0;
    if(node.getMacDeviceTimeAns(&networkTime, &fracSecond, true)) {
      Serial.print(F("[LoRaWAN] DeviceTime Unix:\t"));
      Serial.println(networkTime);
      Serial.print(F("[LoRaWAN] LinkCheck second:\t1/"));
      Serial.println(fracSecond);
    }
  
  } else if(state == RADIOLIB_ERR_RX_TIMEOUT) {
    Serial.println(F("timeout!"));
  
  } else {
    Serial.print(F("failed, code "));
    Serial.println(state);
  }

  // on EEPROM enabled boards, you can save the current session
  // by calling "saveSession" which allows retrieving the session after reboot or deepsleep
  
    node.saveSession();
  

  // wait before sending another packet
  uint32_t minimumDelay = 60000;                  // try to send once every minute
  uint32_t interval = node.timeUntilUplink();     // calculate minimum duty cycle delay (per law!)
	uint32_t delayMs = max(interval, minimumDelay); // cannot send faster than duty cycle allows

  delay(delayMs);
}

[StevenCellist]

@nmaas87 did you wipe between versions? (And reset on console)

[nmaas87]

Yes @StevenCellist - I actually did not and on first tried restore of the "old" 6.3.0 session I got the error

[LoRaWAN] Resuming previous session ... failed, code 20481

However I read that the session structure had changed, so I did already think that would fail :).
I also deleted the build directory to be sure the Arduino would build from scratch and not use some already compiled/half compiled stuff.

[StevenCellist]

So you performed a node.wipe(), then did beginOTAA which returned -1106? What... It's literally worked like a charm on my device which is happily running under .1% dutycycle for five days straight now.

You'll need to provide debug (not verbose) logs, maybe under separate Issue/Discussion, to see if it's something in the stack or on your side.

Quick note that 6.3.0 did work on the surface but actually was not compliant if you'd squint with your eyes at the by now quite verbose debug output. V6.4.1 is the way to go but must work then of course.

[nmaas87]

Thank you for your feeback, I opened up the appropriate issue report to get this worked the right way :)