Looking for BLE logger app

Hey everyone,
I am using nRF Connect to log GYRO data from my nano 33 BLE sense.
The problem with the nRF is that it crashes when trying to record input for more than 3 minutes. The app raises a problem saying " Invalid macro " .
I am looking for a different android app or a software for windows that can record the data the arduino advertise. It should be able to handle data coming at 50-60Hz.
Do you have any suggestions??
Thank you all,
Stam

This question has been asked many times. The issue is that Windows support for BLE for makers is not good. I guess Windows PCs are simply not an interesting platform for BLE. Nobody wants to cycle through the forest with a PC. :slight_smile:
The easiest option is likely to use a second Arduino that reads the data via BLE and then sends it via Serial to the PC. Serial interfaces have been supported on PCs for decades so you can pick a solution that fits your needs.

My pfodApp will log and plot data via BLE. For Nano 33 BLE there is a free pfodDesigner app that generates all the Arduino code for you.
See Arduino NANO 33 Made Easy No Coding required

BUT... 50/60Hz may be a bit fast for how my app uses BLE and continuous connection is not guaranteed.

Other alternatives are Remote High Speed Data Logging using Arduino/GL AR150/Android/pfodApp
Which uses an inexpensive ($25) router to set up a local network and will log 2000 samples/sec

On Windows (via Wifi) you can use Andy to emulate Android and run the pfodApp.

Thank you @Klaus_K and @drmpf !
I ordered another 33 BLE Sense and will try your suggestion @Klaus_K .
@drmpf I tried using the app and watched the videos on Youtube but couldn't understand how to work with it and integrate it with my code. Your post on the Remote High Speed Data Logging seem interesting but I can't see myself getting it to work with the pfodApp.

I am uploading the code and maybe someone here could help me figure out the bottleneck.
The general application is to recognize heelstike of the foot during normal gait.
*The functions at the top (runningAverage and HeelStrikeDetect) are not being used right now because I can't record the data at high enough speed..

The problem:
When I am recording the data on an interval of 20 ms (50Hz) the data is recorded 1:1 (the goal is to always have 1:1 measurements), meaning for each gyroY data-point there is a DeltaT data-point. If I decrease interval value then the recording is missing a data-point every few loops. In the picture I show two consecutively readings of DeltaT, meaning a gyroY reading was missed. BTW I tried debugging the problem and found that when I flip the order of posY.writeValue (writing gyroY before DeltaT) than the skipped data-point is of DeltaT.

Any suggestions on how to improve the code and hopefully achieving higher 1:1 recording frequency are very welcomed :slight_smile:

I am using nRF Connect on a Xiaomi mi 10T PRO.

#include <ArduinoBLE.h>
#include <Arduino_LSM9DS1.h> //By Femme Verbeek 2.0
#include <Wire.h>
// #include "SparkFun_Qwiic_OpenLog_Arduino_Library.h"

//----------------------------------------------------------------------------------------------------------------------
// BLE UUIDs
//----------------------------------------------------------------------------------------------------------------------

#define BLE_UUID_POS_TEST_SERVICE           "0a950ef7-b5c4-4661-822e-5dccbb25b268"
#define BLE_UUID_MILLIS_TEST_SERVICE        "dd4d9077-f135-45c5-8084-45615b33f1cf"
#define BLE_UUID_ANGULAR_VELOCITY           "d374f529-23b8-40d9-92cb-ca7df88a6482" // GYROY
#define BLE_UUID_TIME                       "5b09ccb1-3f8b-4b43-99c8-2844773acd41" // Time UUID
#define BLE_UUID_RESET_MILLIS               "f360eaaf-b719-4758-bf19-59b67af6a661" // VALUE TRIGGER SETTING 

//----------------------------------------------------------------------------------------------------------------------
// BLE
//----------------------------------------------------------------------------------------------------------------------
// Define custom BLE service for position
BLEService posService( BLE_UUID_POS_TEST_SERVICE );
// BLEService MillisService( BLE_UUID_MILLIS_TEST_SERVICE );
BLEStringCharacteristic posY( BLE_UUID_ANGULAR_VELOCITY, BLERead | BLENotify , 24);
// BLEStringCharacteristic MillisCounter( BLE_UUID_TIME, BLERead | BLENotify , 24);
// BLEIntCharacteristic ResetMillisCounter( BLE_UUID_RESET_MILLIS, BLERead | BLEWrite);

const int BLE_LED_PIN = LED_BUILTIN;
const int RSSI_LED_PIN = LED_PWR;


// OpenLog myLog; //Create instance
// String myFile;

static int i=0;
static int AVERAGELEN=4; //Average array length
static double AngleThreshold=10; //Degree
static double GyroThreshold=6; //
static bool AngleTriggered=false;
static bool ZeroCrossP2M=false; //Flag for crossing from plus to minus, expecting heel strike
static double LM[4];      // LastMeasurements array
static double sum; //Current average
static double AvgArray[4]; //Average array
static int count;
static int currIndex;
static float AngleAccumulation;

double runningAverage(double NewgyroY)
{
  // keep sum updated to improve speed.
  
  sum = AvgArray[AVERAGELEN-1] - LM[currIndex];
  LM[currIndex] = NewgyroY;
  sum += LM[currIndex];
  currIndex = currIndex + 1;
  currIndex = currIndex % AVERAGELEN;
  if (count < AVERAGELEN)
  {
    count++; //Valid during the beginning of the walk
  } 
  for (i=0; i<AVERAGELEN-2; i++) //Moving average array
  {  
    AvgArray[i+1] = AvgArray[i];
  }
  AvgArray[0] = sum / count;
  return AvgArray[0]; 
}

bool HeelStrikeDetect(double runningAvgGyroY, intDeltaT)
{ 
  
  if (ZeroCrossP2M)
  {
    if (AvgArray[0] > AvgArray[1]) //This means that the previous measurement was the trough aka heel strike
    //assuming the measurements are taken in the order of 200 points per step, it should me good enough. 
    //TODO:I should calculate this
    {
      ZeroCrossP2M = false;
      return true;
    }
  }
  if (AvgArray[0] <= 0  &&  AvgArray[1] > 0)
  {
    ZeroCrossP2M = true;
    return false;
  }

  // if (runningAvgGyroY > 0)
  // { //Leg is swinging forward
  //   AngleAccumulation += runningAvgGyroY * DeltaT * 1000;
  //   if (AngleAccumulation >= AngleThreshold)
  //   {
  //     AngleAccumulation = 0;
  //     AngleTriggered = true;
  //   }
  // }
  // else AngleAccumulation = 0;

  // if (AngleTriggered)
  // {
  //   if (runningAvgGyroY <= GyroThreshold)
  //   {
  //     AngleTriggered = false;
  //     return true;
  //   }
  // }
}

void setup() 
{
  // Initialize internal LED (for visual debugging)
  Serial.begin(115200);
  Wire.begin();
  Wire.setClock(400000);
  pinMode( BLE_LED_PIN, OUTPUT );
  pinMode( RSSI_LED_PIN, OUTPUT );
  
  // // Create a file
  // myFile = ("log.txt");
  // myLog.begin(); //Open connection to OpenLog
  // myLog.append(myFile); 
  // myLog.syncFile();
  
  // Initialize and Setup IMU
  if (!IMU.begin())
  { Serial.println("Failed to initialize IMU!");
    while (1);
  }

  // Accelerometer code
    IMU.setAccelFS(2);
    IMU.setAccelODR(4);
    IMU.setAccelOffset(-0.007951, -0.026658, -0.008210);
    IMU.setAccelSlope (0.994678, 0.987083, 0.998473);
    // Gyroscope code
    IMU.setGyroFS(1);
    IMU.setGyroODR(4);
    IMU.setGyroOffset (0.583221, 0.534546, 1.205063);
    IMU.setGyroSlope (1.187217, 1.134015, 1.156773);
    // Magnetometer code
    IMU.setMagnetFS(0);
    IMU.setMagnetODR(8);
    IMU.setMagnetOffset(2.937012, 59.017334, 33.474731);
    IMU.setMagnetSlope (4.640753, 4.727273, 4.741902);

  /*****************************************************************************************************************************
  *********  FS  Full Scale       setting 0: ±245°/s | 1: ±500°/s | 2: ±1000°/s | 3: ±2000°/s       ****************************
  *********  ODR Output Data Rate setting 0:off | 1:10Hz | 2:50Hz | 3:119Hz | 4:238Hz | 5:476Hz, (not working 6:952Hz)   *******
  *****************************************************************************************************************************/
  IMU.gyroUnit = DEGREEPERSECOND; //   DEGREEPERSECOND  RADIANSPERSECOND  REVSPERMINUTE  REVSPERSECOND
  if ( setupBleMode() )
  {
    digitalWrite( BLE_LED_PIN, HIGH );
  }

  // Runningavg setup //for later use
  sum = 0;
  count = 0;
  currIndex = 0;
  for (int i=0; i<AVERAGELEN; i++){ //Initiating the lastmeasurements average array
    LM[i]=0;
    AvgArray[i]=0;
  }
}

void loop() 
{
  static long previousMillis = 0;
  // myLog.append(myFile);
  int DeltaT=0;
  // listen for BLE peripherals to connect:
  BLEDevice central = BLE.central();

  if (central) {
    //Serial.print("Connected to central: ");
    //Serial.println(central.address());

    while (central.connected()) {
      if (0) {
        DeltaT=0;
        // MillisCounter.writeValue(String(DeltaT)); //Resetting the Delta
      }

      int interval = 20; // 1000 / 20 = 50Hz
      unsigned long currentMillis = millis();
      
      if (currentMillis - previousMillis > interval) {
        DeltaT += currentMillis - previousMillis;
        previousMillis = currentMillis;

        if (central.rssi() != 0) {
          digitalWrite(RSSI_LED_PIN, LOW);
          float gyroX, gyroY, gyroZ;
          if (IMU.gyroAvailable()) {
            IMU.readGyro(gyroX, gyroY, gyroZ);
            posY.writeValue(String(DeltaT));
            // double runningAvgGyroY = runningAverage(gyroY); //for later use
            posY.writeValue(String(gyroY));
            
            // MillisCounter.writeValue(String(DeltaT));
            // Serial.println(gyroY);
            // Serial.println(DeltaT);
            // myLog.append(myFile);
            // myLog.print(F("Gyro: Y:"));
            // myLog.print(gyroY);
            // myLog.print(";");
            // myLog.println(DeltaT);
            // bool HSDetected=HeelStrikeDetect(runningAvgGyroY, DeltaT); //for later use
          }
          
        } else {
          digitalWrite(RSSI_LED_PIN, HIGH);
        }
      } 
    } 


  } 
}  


bool setupBleMode()
{
  if ( !BLE.begin() )
  {
    return false;
  }

  // set advertised local name and service UUID:
  BLE.setDeviceName( "Arduino Nano 33 BLE" );
  BLE.setLocalName( "Arduino Nano 33 BLE" );
  BLE.setAdvertisedService( posService );
  // BLE.setAdvertisedService( MillisService );

  // BLE add characteristics
  posService.addCharacteristic( posY );
  // MillisService.addCharacteristic( MillisCounter );
  // MillisService.addCharacteristic( ResetMillisCounter );

  // add service
  BLE.addService( posService );
  // BLE.addService( MillisService );

  // set the initial value for the characteristic:
  posY.writeValue( String(0.0) );
  // MillisCounter.writeValue( String(0) );
  // ResetMillisCounter.writeValue(0);
  // set the interval for the communication
  BLE.setConnectionInterval(0x0001, 0x0001); // 1/(1 * 1.25ms) 800 Hz
  // start advertising
  BLE.advertise();

  return true;
}

GyroBLECalibratedAndSave2SDAndAlgoForum.ino (8.1 KB)

Here is some sample code using pfod's BleSerial to send text data
My IMU library did not have most of your methods ??
GyroBLECalibratedAndSave2SDAndAlgoForum_Rev1.ino (14.6 KB)
The output sent to Nordic nRF UART app is

20,1.65,0.37,0.31
21,1.65,0.37,0.37
20,1.40,0.18,0.24
20,1.28,0.18,0.24
20,1.53,0.43,0.43
22,1.65,0.37,0.24
23,1.40,0.31,0.37
20,1.34,0.24,0.37
20,1.40,0.43,0.31
20,1.46,0.12,0.24
20,1.46,0.31,0.24
20,1.46,0.24,0.37
20,1.40,0.37,0.18
20,1.28,0.31,0.37
20,1.40,0.37,0.31
20,1.53,0.37,0.31
20,1.59,0.31,0.31
1 Like

To get the highest data throughput you should not use String characteristics but raw data. Strings are a wasteful way to send values. Each character has 8 bits which allows for 256 values, but you use only 10 (numbers 0-9).

If you have multiple data values, you can use a compound array. With Strings you need a separator character which is an additional byte wasted for each. Sending raw data, you do not need any separation just a predefined structure.

For example, 4 x 16-bit numbers as String need 23 bytes vs 8 bytes when sending raw values.

For a real smart sensor, you would not need anywhere near the data rate. You would do the calculation in the sensor and then only transfer smart data. e.g., total strike counts or something like that. That would be one value every second.

I no longer use the rssi check in my code. I developed that as a fix for an issue that crashed the BLE stack. This has been fixed a while ago. I do not remember which version of the ArduinoBLE library.

1 Like

@drmpf Thank you very much, it runs well. Is the BLE.setConnectionInterval(80, 160); // 160 => 200mS max connection interval to match default pfodBLEBufferedSerial setting a constraint that can be overcome?

@Klaus_K As for the "a real smart sensor, you would not need anywhere near the data rate." I disagree :slight_smile: The device I am working on is real smart and needs the data rate, for reference commercial high end motion detection systems are recording at much higher rate (XSENS up to 240 Hz). The data I record will be segmented into the 7 phase of gait, 60 Hz is really the minimum. This is just the beginning of my thesis. I agree that for measuring heel strike even 5-10 Hz would be enough following nyquist role.

Should I omit the rssi condition in the code?
I tried sending the raw data by changing to BLEFloatCharacteristic instead of String. On the central using nRF connect I couldn't translate the data to make sense :frowning: Would you please help me with advertising data that can be translated into digits? Maybe you have an example you wrote? I assume char is taking the least amount of bytes for digits, but float should also be more efficient than string.

Many thanks, Nitzan

The bleBufferedSerial is not used now see the line
// bleBufferedSerial.connect(&bleSerial);
in setup. The output goes directly to the bleSerial and is sent as soon as the 20byte buffer fills.
By all means try playing with connection Interval, but if it works now . . .

Yes, you can.

I tried a couple of different generic BLE apps. None of them format the data to match generic data types. The BLE specification even allows to add descriptors to each characteristic to tell the client the exact format. Some apps even show the descriptors, but they ignore them.

Any BLE data can be translated into digits. What is your endgame here? Do you want to write your own app? What do you want to do with the data e.g., store it for analysis, drive a display or ...?
Where do you need the data, on a phone, tablet, PC, Raspberry Pi ... ?

How do you use nRF Connect? Do you just look at characteristics and services? Any feature you like.

I think the application can be split into two parts: online and offline.
-Online would be to compute/recognize (on-board) heelstike and stride length, according to those values an infrared diode will be activated.
-Offline would be to verify the performance of the on-board computation and algorithm. The infrared diode is for visual analysis as well.

I am not planning on building an app, after all I am Mechanical eng and soon MS.c in Biomed eng :slight_smile:

For the online processes I need to work with digits. For the offline analysis, receiving a form of data that is not digits (e.g. hex, dec or something else) and being able to convert it to digits is perfectly fine.

The way I use nRF Connect is by recording the received data on my phone, exporting it to .xml and converting it to excel form on the computer, later I will be importing xml to Matlab and analysing it there. I only look at the "value-string" column.
Screenshot 2021-06-08 150608

It does not look like I can do that in the iOS version. I can only save a log file with all kinds of debug data. Can you post an small xml example file with float values?

How do you convert the XML to Excel?

How many data points do you need for offline analysis? And how many values and types do you need per data point. e.g. 1 float + 2 32-bit int ...
The Arduino Nano 33 BLE has a lot of RAM. You could store a session inside a RAM buffer and transfer the data slow enough for nRF Connect to capture it.

Very good idea

@Klaus_K Here is a screen recording from my phone.
I also upload both the xml and the xlsx files. To convert xml to xlsx you open the xml using excel and excel will ask you if you want to open it as a xml table, you click "yes" and that is it.
Dropbox link

For offline analysis I need at least 15 seconds (with respect to minimum 60Hz) which is 900 data points, I would prefer higher rate over longer period.
I am not sure what you mean by types per data point, for the values range gyroY should range -300 +300 and DeltaT should be 0-15000 (for 15 seconds). Is this what you meant?

The updated code with Float is:

#include <ArduinoBLE.h>
#include <Arduino_LSM9DS1.h> //By Femme Verbeek 2.0
#include <Wire.h>
// #include "SparkFun_Qwiic_OpenLog_Arduino_Library.h"

//----------------------------------------------------------------------------------------------------------------------
// BLE UUIDs
//----------------------------------------------------------------------------------------------------------------------

#define BLE_UUID_POS_TEST_SERVICE           "0a950ef7-b5c4-4661-822e-5dccbb25b268"
#define BLE_UUID_MILLIS_TEST_SERVICE        "dd4d9077-f135-45c5-8084-45615b33f1cf"
#define BLE_UUID_ANGULAR_VELOCITY           "d374f529-23b8-40d9-92cb-ca7df88a6482" // GYROY
#define BLE_UUID_TIME                       "5b09ccb1-3f8b-4b43-99c8-2844773acd41" // Time UUID
#define BLE_UUID_RESET_MILLIS               "f360eaaf-b719-4758-bf19-59b67af6a661" // VALUE TRIGGER SETTING 

//----------------------------------------------------------------------------------------------------------------------
// BLE
//----------------------------------------------------------------------------------------------------------------------
// Define custom BLE service for position
BLEService posService( BLE_UUID_POS_TEST_SERVICE );
// BLEService MillisService( BLE_UUID_MILLIS_TEST_SERVICE );
BLEFloatCharacteristic posY( BLE_UUID_ANGULAR_VELOCITY, BLERead | BLENotify);
// BLEStringCharacteristic MillisCounter( BLE_UUID_TIME, BLERead | BLENotify , 24);
// BLEIntCharacteristic ResetMillisCounter( BLE_UUID_RESET_MILLIS, BLERead | BLEWrite);

const int BLE_LED_PIN = LED_BUILTIN;
const int RSSI_LED_PIN = LED_PWR;


// OpenLog myLog; //Create instance
// String myFile;

static int i=0;
static int AVERAGELEN=4; //Average array length
static double AngleThreshold=10; //Degree
static double GyroThreshold=6; //
static bool AngleTriggered=false;
static bool ZeroCrossP2M=false; //Flag for crossing from plus to minus, expacting heel strike
static double LM[4];      // LastMeasurements array
static double sum; //Current average
static double AvgArray[4]; //Average array
static int count;
static int currIndex;
static float AngleAccumulation;

double runningAverage(double NewgyroY)
{
  // keep sum updated to improve speed.
  
  sum = AvgArray[AVERAGELEN-1] - LM[currIndex];
  LM[currIndex] = NewgyroY;
  sum += LM[currIndex];
  currIndex = currIndex + 1;
  currIndex = currIndex % AVERAGELEN;
  if (count < AVERAGELEN)
  {
    count++; //Valid during the begining of the walk
  } 
  for (i=0; i<AVERAGELEN-2; i++) //Moving average array
  {  
    AvgArray[i+1] = AvgArray[i];
  }
  AvgArray[0] = sum / count;
  return AvgArray[0]; 
}

bool HeelStrikeDetect(double runningAvgGyroY, long DeltaT)
{ 
  
  if (ZeroCrossP2M)
  {
    if (AvgArray[0] > AvgArray[1]) //This means that the previus measurement was the trough aka heel strike
    //assuming the measurements are taken in the order of 200 points per step, it should me good enough. 
    //TODO:I should calculate this
    {
      ZeroCrossP2M = false;
      return true;
    }
  }
  if (AvgArray[0] <= 0  &&  AvgArray[1] > 0)
  {
    ZeroCrossP2M = true;
    return false;
  }

  // if (runningAvgGyroY > 0)
  // { //Leg is swinging forward
  //   AngleAccumulation += runningAvgGyroY * DeltaT * 1000;
  //   if (AngleAccumulation >= AngleThreshold)
  //   {
  //     AngleAccumulation = 0;
  //     AngleTriggered = true;
  //   }
  // }
  // else AngleAccumulation = 0;

  // if (AngleTriggered)
  // {
  //   if (runningAvgGyroY <= GyroThreshold)
  //   {
  //     AngleTriggered = false;
  //     return true;
  //   }
  // }
}

void setup() 
{
  // Initialize internal LED (for visual debugging)
  Serial.begin(115200);
  Wire.begin();
  Wire.setClock(400000);
  pinMode( BLE_LED_PIN, OUTPUT );
  pinMode( RSSI_LED_PIN, OUTPUT );
  
  // // Create a file
  // myFile = ("log.txt");
  // myLog.begin(); //Open connection to OpenLog
  // myLog.append(myFile); 
  // myLog.syncFile();
  
  // Initialize and Setup IMU
  if (!IMU.begin())
  { Serial.println("Failed to initialize IMU!");
    while (1);
  }

  // Accelerometer code
    IMU.setAccelFS(2);
    IMU.setAccelODR(4);
    IMU.setAccelOffset(-0.007951, -0.026658, -0.008210);
    IMU.setAccelSlope (0.994678, 0.987083, 0.998473);
    // Gyroscope code
    IMU.setGyroFS(1);
    IMU.setGyroODR(4);
    IMU.setGyroOffset (0.583221, 0.534546, 1.205063);
    IMU.setGyroSlope (1.187217, 1.134015, 1.156773);
    // Magnetometer code
    IMU.setMagnetFS(0);
    IMU.setMagnetODR(8);
    IMU.setMagnetOffset(2.937012, 59.017334, 33.474731);
    IMU.setMagnetSlope (4.640753, 4.727273, 4.741902);

  /*****************************************************************************************************************************
  *********  FS  Full Scale       setting 0: ±245°/s | 1: ±500°/s | 2: ±1000°/s | 3: ±2000°/s       ****************************
  *********  ODR Output Data Rate setting 0:off | 1:10Hz | 2:50Hz | 3:119Hz | 4:238Hz | 5:476Hz, (not working 6:952Hz)   *******
  *****************************************************************************************************************************/
  IMU.gyroUnit = DEGREEPERSECOND; //   DEGREEPERSECOND  RADIANSPERSECOND  REVSPERMINUTE  REVSPERSECOND
  if ( setupBleMode() )
  {
    digitalWrite( BLE_LED_PIN, HIGH );
  }

  // Runningavg setup //for later use
  sum = 0;
  count = 0;
  currIndex = 0;
  for (int i=0; i<AVERAGELEN; i++){ //Initiating the lastmeasurements average array
    LM[i]=0;
    AvgArray[i]=0;
  }
}

void loop() 
{
  static long previousMillis = 0;
  // myLog.append(myFile);
  int DeltaT=0;
  // listen for BLE peripherals to connect:
  BLEDevice central = BLE.central();

  if (central) {
    //Serial.print("Connected to central: ");
    //Serial.println(central.address());

    while (central.connected()) {
      if (0) {
        DeltaT=0;
        // MillisCounter.writeValue(String(DeltaT)); //Resetting the Delta
      }

      int interval = 10; // 1000 / 20 = 50Hz
      unsigned long currentMillis = millis();
      
      if (currentMillis - previousMillis > interval) {
        DeltaT += currentMillis - previousMillis;
        previousMillis = currentMillis;

        if (central.rssi() != 0) {
          digitalWrite(RSSI_LED_PIN, LOW);
          float gyroX, gyroY, gyroZ;
          if (IMU.gyroAvailable()) {
            IMU.readGyro(gyroX, gyroY, gyroZ);
            posY.writeValue(DeltaT);
            // double runningAvgGyroY = runningAverage(gyroY); //for later use
            posY.writeValue(gyroY);
            
            // MillisCounter.writeValue(String(DeltaT));
            // Serial.println(gyroY);
            // Serial.println(DeltaT);
            // myLog.append(myFile);
            // //myLog.print(F("Gyro: Y:"));
            // myLog.print(gyroY);
            // myLog.print(";");
            // myLog.println(DeltaT);
            // bool HSDetected=HeelStrikeDetect(runningAvgGyroY, DeltaT); //for later use
          }
          
        } else {
          digitalWrite(RSSI_LED_PIN, HIGH);
        }
      } 
    } 


  } 
}  


bool setupBleMode()
{
  if ( !BLE.begin() )
  {
    return false;
  }

  // set advertised local name and service UUID:
  BLE.setDeviceName( "Arduino Nano 33 BLE" );
  BLE.setLocalName( "Arduino Nano 33 BLE" );
  BLE.setAdvertisedService( posService );
  // BLE.setAdvertisedService( MillisService );

  // BLE add characteristics
  posService.addCharacteristic( posY );
  // MillisService.addCharacteristic( MillisCounter );
  // MillisService.addCharacteristic( ResetMillisCounter );

  // add service
  BLE.addService( posService );
  // BLE.addService( MillisService );

  // set the initial value for the characeristic:
  posY.writeValue(0.0);
  // MillisCounter.writeValue( String(0) );
  // ResetMillisCounter.writeValue(0);
  // set the interval for the communication
  BLE.setConnectionInterval(0x0001, 0x0001); // 1/(1 * 1.25ms) 800 Hz
  // start advertising
  BLE.advertise();

  return true;
}

Thanks. That was helpful. nRF Connect seems to have some more features on Android indeed.

I played a bit with your data and Excel and here is what I found and a few questions and suggestions.

  • It is possible to convert the data in Excel into floating point numbers.

  • Before any of the conversion can be done the byte order needs to be verified/changed. Most ARM processors are little-endian implementations.
    -- e.g., 0x12345678 is stored in RAM 0x78 first and 0x12 last and therefore send over BLE in that order
    -- nRF connect will present the bytes in the order received
    -- the UUID will look OK because nRF knows the order and therefore it will look like in your code

  • Regarding the number format it might be better to convert the numbers on the Arduino into fixed point data.
    -- e.g., instead of sending 0.12345f you can send 0.12345 * 1,000,000 = 123450
    -- This will make the conversion a bit easier in Excel and it may make it easier for you to spot issues while looking at the raw data

  • We could also try reversing the byte order already on the Arduino. CMSIS has an intrinsic for that called __REV.

  • In your example you send DeltaT and gyroY to the same characteristic. That will make it necessary to filter the data in Excel.

  • I believe you do not need to send DeltaT. If the samples are all equidistant you can simply count. I suspect you would see in the gyroY data if any samples are missing.

  • Regarding the buffer, let’s say we use 100kB and only save gyroY as 32-bit that would give you 25k buffer. At 60HZ you get 426 seconds or around 7 Minutes.

  • If you really wanted to, you could buffer the data while you send. This would give you some extra time or allow you to send other values as well.

  • I changed connection handling in my sketches from the way I used to in the example you used. I can provide you with an updated example if you like. I think it looks cleaner it makes life easier.

Let me know what you think.

So if we REV the byte order before sending and then nRF REV the order (because of how it is received). Do we then get the right order to read? is it like minus*minus = plus ?
.

Is this process for using int instead of float? I am happy with whatever makes the function reliable and faster.
.

I do need the delta between each measurement, It is useful for step length and timing, I don't want to rely on frequency for timing. If we could advertise "gyroY;DeltaT" as one package then we get 1:1 data points and any known separation character ( ; Semicolon in this case) can be easily identify on Matlab and excel and be used to indicate two distinct value types.
.

Each recording session can be limited to 20 sec because I am also constrained by the length of the lab and coverage of motion detection cameras. I don't know the calculation of buffer size but if we can get recordings of 20 seconds at 60-100Hz that is amazing. Can it be stored on the nano and be sent at the end of each session?
I would love an updated example.

Your help is very appreciated! I can't stress it enough, thank you :slight_smile:

Yes, you can try with the example below.

We are not going to use a separation character. We send raw data. Do not worry, it will work out nicely.

OK, I will adapt the example to send both.

That should be easy.

Here is an example of just the new way of handling BLE and the byte reversal. I kept the example generic and used the ADC as sensor in the hope others might find it useful as well.
Can you test this with nRF connect and then convert the data in Excel?

=HEX2DEC( A1 )

It should give you values between 0 and 4095 e.g. 12-bit ADC.

/*
  This example creates a BLE peripheral with a Sensor Service.

  The circuit:
  - Arduino Nano 33 BLE and BLE Sense.

  You can use a generic BLE central app, like LightBlue (iOS and Android) or
  nRF Connect (Android), to interact with the services and characteristics
  created in this sketch.

  This example code is in the public domain.
*/


#include <ArduinoBLE.h>

//----------------------------------------------------------------------------------------------------------------------
// BLE UUIDs
//----------------------------------------------------------------------------------------------------------------------

#define BLE_UUID_SENSOR_SERVICE                   "19120000-1713-4AE2-5A50-946B02A2FFAF"
#define BLE_UUID_SENSOR                           "19120001-1713-4AE2-5A50-946B02A2FFAF"

//----------------------------------------------------------------------------------------------------------------------
// BLE
//----------------------------------------------------------------------------------------------------------------------

#define BLE_DEVICE_NAME                           "Arduino Nano 33 BLE"
#define BLE_LOCAL_NAME                            "Arduino Nano 33 BLE"

BLEService sensorService( BLE_UUID_SENSOR_SERVICE );
BLEUnsignedLongCharacteristic sensorCharacteristic( BLE_UUID_SENSOR, BLERead | BLENotify );

//----------------------------------------------------------------------------------------------------------------------
// APP & I/O
//----------------------------------------------------------------------------------------------------------------------

// Reversing Byte order allows uint32_t data to be converted in Excel with HEX2DEC function
#define REVERSE_BYTE_ORDER

typedef struct __attribute__( ( packed ) )
{
  uint32_t value;
  bool updated;
} sensor_data_t;

sensor_data_t sensorData = { .value = 0, .updated = false };

#define BLE_LED_PIN                               LED_BUILTIN
#define SENSOR_PIN                                A0

void setup()
{
  Serial.begin( 9600 );
  while ( !Serial );

  pinMode( BLE_LED_PIN, OUTPUT );
  digitalWrite( BLE_LED_PIN, LOW );

  analogReadResolution( 12 );

  if ( !setupBleMode() )
  {
    Serial.println( "Failed to initialize BLE!" );
    while ( 1 );
  }
  else
  {
    Serial.println( "BLE initialized. Waiting for clients to connect." );
  }
}


void loop()
{
  bleTask();
  sensorTask();
}


void sensorTask()
{
#define SENSOR_UPDATE_INTERVAL 10
  static uint32_t previousMillis = 0;

  uint32_t currentMillis = millis();
  if ( currentMillis - previousMillis >= SENSOR_UPDATE_INTERVAL )
  {
    previousMillis = currentMillis;
    sensorData.value = analogRead( SENSOR_PIN );
    #ifdef REVERSE_BYTE_ORDER
    sensorData.value = __REV( sensorData.value );
    #endif
    sensorData.updated = true;
  }
}

bool setupBleMode()
{
  if ( !BLE.begin() )
  {
    return false;
  }

  // set advertised local name and service UUID
  BLE.setDeviceName( BLE_DEVICE_NAME );
  BLE.setLocalName( BLE_LOCAL_NAME );
  BLE.setAdvertisedService( sensorService );

  // BLE add characteristics
  sensorService.addCharacteristic( sensorCharacteristic );

  // add service
  BLE.addService( sensorService );

  // set the initial value for the characeristic
  sensorCharacteristic.writeValue( sensorData.value );

  // set BLE event handlers
  BLE.setEventHandler( BLEConnected, blePeripheralConnectHandler );
  BLE.setEventHandler( BLEDisconnected, blePeripheralDisconnectHandler );

  // start advertising
  BLE.advertise();

  return true;
}


void bleTask()
{
#define BLE_UPDATE_INTERVAL 10
  static uint32_t previousMillis = 0;

  uint32_t currentMillis = millis();
  if ( currentMillis - previousMillis >= BLE_UPDATE_INTERVAL )
  {
    previousMillis = currentMillis;
    BLE.poll();
  }

  if ( sensorData.updated )
  {
    sensorData.updated = false;
    sensorCharacteristic.writeValue( sensorData.value );
  }
}


void blePeripheralConnectHandler( BLEDevice central )
{
  digitalWrite( BLE_LED_PIN, HIGH );
  Serial.print( F( "Connected to central: " ) );
  Serial.println( central.address() );
}


void blePeripheralDisconnectHandler( BLEDevice central )
{
  digitalWrite( BLE_LED_PIN, LOW );
  Serial.print( F( "Disconnected from central: " ) );
  Serial.println( central.address() );
}

The conversion function works great in Excel, I collected the data on my phone (with nRF Connect) using the same process as I posted earlier. The N column is the conversion of M column.

I commented while ( !Serial ); because I also wanted to make sure we can communicate while connected to external battery.

The code looks neat :slight_smile:

:+1:

Here is the example with the buffer.

BLE Sensor Example with data buffer (Click to show)
/*
  This example creates a BLE peripheral with a Sensor Service.
  The data is collected in a buffer and can be replayed over BLE at a slower speed.

  The circuit:
  - Arduino Nano 33 BLE and BLE Sense.

  You can use a generic BLE central app, like BLE Scanner (iOS and Android) or
  nRF Connect (iOS and Android), to interact with the services and characteristics
  created in this sketch.

  This example code is in the public domain.
*/


#include <ArduinoBLE.h>

//----------------------------------------------------------------------------------------------------------------------
// BLE UUIDs
//----------------------------------------------------------------------------------------------------------------------

#define BLE_UUID_SENSOR_SERVICE                   "19120000-1713-4AE2-5A50-946B02A2FFAF"
#define BLE_UUID_SENSOR                           "19120001-1713-4AE2-5A50-946B02A2FFAF"
#define BLE_UUID_SENSOR_CONTROL                   "19120002-1713-4AE2-5A50-946B02A2FFAF"

#define BLE_UUID_CHARACTERISTIC_USER_DESCRIPTION  "2901"

//----------------------------------------------------------------------------------------------------------------------
// BLE
//----------------------------------------------------------------------------------------------------------------------

typedef struct __attribute__( ( packed ) )
{
  uint32_t position;
  uint32_t time;
} sensor_data_t;

typedef union
{
  sensor_data_t values;
  uint8_t bytes[ sizeof( sensor_data_t ) ];
} sensor_data_ut;

sensor_data_ut sensorData;
bool sensorDataUpdated = false;

#define BLE_DEVICE_NAME                           "Arduino Nano 33 BLE"
#define BLE_LOCAL_NAME                            "Arduino Nano 33 BLE"

BLEService sensorService( BLE_UUID_SENSOR_SERVICE );
BLECharacteristic sensorCharacteristic( BLE_UUID_SENSOR, BLERead | BLENotify, sizeof( sensor_data_ut ) );
BLEByteCharacteristic controlCharacteristic( BLE_UUID_SENSOR_CONTROL, BLERead | BLEWrite );

BLEDescriptor controlDescriptor( BLE_UUID_CHARACTERISTIC_USER_DESCRIPTION, "CMD:1-RESET,2-RECORD,3-REPLAY" );

//----------------------------------------------------------------------------------------------------------------------
// APP & I/O
//----------------------------------------------------------------------------------------------------------------------

// Reversing Byte order allows uint32_t data to be converted in Excel with HEX2DEC function
#define REVERSE_BYTE_ORDER

#define DATA_BUFFER_SIZE     (20 * 100)

enum BUFER_STATE_TYPE { BUFFER_STATE_READY,
                        BUFFER_STATE_RECORD,
                        BUFFER_STATE_REPLAY,
                        BUFFER_STATE_FULL
                      };

typedef struct __attribute__( ( packed ) )
{
  sensor_data_t values[ DATA_BUFFER_SIZE ];
  uint32_t index;
  uint32_t state = BUFFER_STATE_READY;
} sensor_buffer_t;

sensor_buffer_t dataBuffer;
bool sensorActive = false;


#define BLE_LED_PIN                               LED_BUILTIN
#define BUFFER_LED_PIN                            LED_PWR
#define SENSOR_PIN                                A0

void setup()
{
  Serial.begin( 9600 );
  while ( !Serial );

  pinMode( BLE_LED_PIN, OUTPUT );
  digitalWrite( BLE_LED_PIN, LOW );
  pinMode( BUFFER_LED_PIN, OUTPUT );
  digitalWrite( BUFFER_LED_PIN, LOW );

  analogReadResolution( 12 );

  if ( !setupBleMode() )
  {
    Serial.println( "Failed to initialize BLE!" );
    while ( 1 );
  }
  else
  {
    Serial.println( "BLE initialized. Waiting for clients to connect." );
  }
}


void loop()
{
  bleTask();
  sensorTask();
  sendDataTask( 50 );
}


void sensorTask()
{
  const uint32_t SENSOR_SAMPLING_INTERVAL = 10;
  static uint32_t previousMillis = 0;
  
  if ( !sensorActive )
  {
    return;
  }

  uint32_t currentMillis = millis();
  if ( currentMillis - previousMillis >= SENSOR_SAMPLING_INTERVAL )
  {
    previousMillis = currentMillis;
    
    sensorData.values.position = analogRead( SENSOR_PIN );
    sensorData.values.time = currentMillis;

    #ifdef REVERSE_BYTE_ORDER
    sensorData.values.position = __REV( sensorData.values.position );
    sensorData.values.time = __REV( sensorData.values.time );
    #endif
    
    sensorDataUpdated = true;
    
    if ( dataBuffer.state == BUFFER_STATE_RECORD )
    {
      dataBuffer.values[ dataBuffer.index ] = sensorData.values;
      dataBuffer.index = ( dataBuffer.index + 1 ) % DATA_BUFFER_SIZE;
      if ( dataBuffer.index == 0 )
      {
        dataBuffer.state = BUFFER_STATE_FULL;
        digitalWrite( BUFFER_LED_PIN, HIGH );
        sensorActive = false;
        Serial.println( "Recording done" );
      }
    }
  }
}


int32_t sendDataTask( uint32_t interval )
{
  const uint32_t SEND_DELAY_MS = 30000;
  static uint32_t previousMillis = 0;
  static bool startUpDelay = true;

  if ( dataBuffer.state != BUFFER_STATE_REPLAY )
  {
    return -1;
  }
  
  if ( dataBuffer.index == 0 )
  {
    if ( startUpDelay )
    {
      startUpDelay = false;
      previousMillis = millis();
      Serial.println( "Get ready for send" );
      return -1;
    }
    uint32_t currentMillis = millis();
    if ( currentMillis - previousMillis < SEND_DELAY_MS )
    {
      return -1;
    }
    else 
    {  
      startUpDelay = true;
      Serial.println( "Sending data" );
    }
  }

  uint32_t currentMillis = millis();
  if ( currentMillis - previousMillis >= interval )
  {
    previousMillis = currentMillis;
    
    sensorData.values = dataBuffer.values[ dataBuffer.index ];
    sensorDataUpdated = true;
    dataBuffer.index = ( dataBuffer.index + 1 ) % DATA_BUFFER_SIZE;
    if ( dataBuffer.index == 0 )
    {
      dataBuffer.state = BUFFER_STATE_READY;
      digitalWrite( BUFFER_LED_PIN, LOW );
      Serial.println( "Sending done" );
    }
    return dataBuffer.index;
  }
  return -1;
}


bool setupBleMode()
{
  if ( !BLE.begin() )
  {
    return false;
  }

  // set advertised local name and service UUID
  BLE.setDeviceName( BLE_DEVICE_NAME );
  BLE.setLocalName( BLE_LOCAL_NAME );
  BLE.setAdvertisedService( sensorService );

  // BLE add characteristics
  sensorService.addCharacteristic( sensorCharacteristic );
  sensorService.addCharacteristic( controlCharacteristic );

  // BLE add descriptors
  controlCharacteristic.addDescriptor( controlDescriptor );

  // add service
  BLE.addService( sensorService );

  // set the initial value for the characeristic
  sensorCharacteristic.writeValue( sensorData.bytes, sizeof sensorData.bytes );
  controlCharacteristic.writeValue( 0 );

  // set BLE event handlers
  BLE.setEventHandler( BLEConnected, blePeripheralConnectHandler );
  BLE.setEventHandler( BLEDisconnected, blePeripheralDisconnectHandler );
  
  // set service and characteristic specific event handlers
  controlCharacteristic.setEventHandler( BLEWritten, controlCharacteristicWrittenHandler );

  // start advertising
  BLE.advertise();

  return true;
}


void bleTask()
{
  const uint32_t BLE_UPDATE_INTERVAL = 10;
  static uint32_t previousMillis = 0;

  uint32_t currentMillis = millis();
  if ( currentMillis - previousMillis >= BLE_UPDATE_INTERVAL )
  {
    previousMillis = currentMillis;
    BLE.poll();
  }

  if ( sensorDataUpdated )
  {
    sensorDataUpdated = false;
    sensorCharacteristic.writeValue( sensorData.bytes, sizeof sensorData.bytes );
  }
}

enum CMD_TYPE { CMD_RESET_BUFFER = 1,
                CMD_RECORD,
                CMD_REPLAY
              };


void controlCharacteristicWrittenHandler( BLEDevice central, BLECharacteristic bleCharacteristic )
{
  Serial.print( "BLE characteristic written. UUID: " );
  Serial.print( bleCharacteristic.uuid() );

  uint8_t cmd;
  if ( bleCharacteristic.uuid() == ( const char* ) BLE_UUID_SENSOR_CONTROL )
  {
    bleCharacteristic.readValue( cmd );
    Serial.print( " CMD: " );
    Serial.print( cmd, HEX );
    switch ( cmd )
    {
      case CMD_RESET_BUFFER:
        dataBuffer.index = 0;
        dataBuffer.state = BUFFER_STATE_READY;
        digitalWrite( BUFFER_LED_PIN, LOW );
        Serial.println( " Reset" );
        break;
      case CMD_RECORD:
        dataBuffer.index = 0;
        dataBuffer.state = BUFFER_STATE_RECORD;
        digitalWrite( BUFFER_LED_PIN, LOW );
        sensorActive = true;
        Serial.println( " Record" );
        break;
      case CMD_REPLAY:
        dataBuffer.index = 0;
        dataBuffer.state = BUFFER_STATE_REPLAY;
        digitalWrite( BUFFER_LED_PIN, LOW );
        sensorActive = false;
        Serial.println( " Replay" );
        break;
        
      default:
        Serial.println( " unknown" );
        break;
    }
  }
}


void blePeripheralConnectHandler( BLEDevice central )
{
  digitalWrite( BLE_LED_PIN, HIGH );
  Serial.print( F( "Connected to central: " ) );
  Serial.println( central.address() );
}


void blePeripheralDisconnectHandler( BLEDevice central )
{
  digitalWrite( BLE_LED_PIN, LOW );
  Serial.print( F( "Disconnected from central: " ) );
  Serial.println( central.address() );
}

It has the following additional features over the previous example

  • two values in one characteristic
  • new characteristic to start collecting data and replay the data at slower speed
    -- Write 01 to reset, 02 to start recording, 03 to replay
  • delay 30 seconds when starting replay, this will allow you to start recording to XML in the app, change as needed
  • the data will be transmitted while recording but stops when the buffer is full

In Excel you can split the column with the following steps

  • make column Format Text
  • mark column
  • go to Data -> Text to Columns
  • Choose option "Fixed width" Next
  • Create break line in the middle Finish
  • then convert the columns separately like before

or use the following formulas without splitting ( you might need a comma instead of semicolon in your Excel version)

= HEX2DEC( LEFT( A1; 8 ) ) // This will extract and convert sensor value
= HEX2DEC( RIGHT( A1; 8 ) ) // This will extract and convert millis value