Hi all!
I'm using the Arduino Nano 33 BLE Sense Rev2 on a project for work for gesture recognition, and I'm going through the tinyML tutorial because it matches what I'm trying to do. I've gotten through the tutorial and I'm at the last part, which is using the model that I have to recognize gestures. The library I'm using is the Arduino TensorFlowLite 2.4.0 ALPHA version which was used when the tutorial was out before it was removed from the Arduino libraries by tensor. All of that in mind, the code is not compiling and its getting caught on some major parts which are making it not work. I've tried a couple different versions of the tensorflow libraries, and have reworked the code so the file paths are matching the code and verified the directories match what it says in the code. I've attached the error message I'm currently working through, but I'd assume its going to get caught on a couple more parts of the code.
The code provided is slightly modified for the BLE Sense rev2 due to to the different onboard IMU, and the missing part in the file path for one of the include statements, but besides that all of the code is straight from the tutorial page. The library is unedited, and the model.h file is included. I'm very much stuck on what to do next, so if anyone has any suggestions on ways to proceed I'd really appreciate it.
#include <Arduino_BMI270_BMM150.h>
/*
IMU Classifier
This example uses the on-board IMU to start reading acceleration and gyroscope
data from on-board IMU, once enough samples are read, it then uses a
TensorFlow Lite (Micro) model to try to classify the movement as a known gesture.
Note: The direct use of C/C++ pointers, namespaces, and dynamic memory is generally
discouraged in Arduino examples, and in the future the TensorFlowLite library
might change to make the sketch simpler.
The circuit:
- Arduino Nano 33 BLE or Arduino Nano 33 BLE Sense board.
Created by Don Coleman, Sandeep Mistry
Modified by Dominic Pajak, Sandeep Mistry
This example code is in the public domain.
*/
#include <TensorFlowLite.h>
#include <tensorflow/lite/micro/all_ops_resolver.h>
#include <tensorflow/lite/micro/tflite_bridge/micro_error_reporter.h>
#include <tensorflow/lite/micro/micro_interpreter.h>
#include <tensorflow/lite/schema/schema_generated.h>
#include "model.h"
const float accelerationThreshold = 2.5; // threshold of significant in G's
const int numSamples = 119;
int samplesRead = numSamples;
// global variables used for TensorFlow Lite (Micro)
tflite::MicroErrorReporter tflErrorReporter;
// pull in all the TFLM ops, you can remove this line and
// only pull in the TFLM ops you need, if would like to reduce
// the compiled size of the sketch.
tflite::AllOpsResolver tflOpsResolver;
const tflite::Model* tflModel = nullptr;
tflite::MicroInterpreter* tflInterpreter = nullptr;
TfLiteTensor* tflInputTensor = nullptr;
TfLiteTensor* tflOutputTensor = nullptr;
// Create a static memory buffer for TFLM, the size may need to
// be adjusted based on the model you are using
constexpr int tensorArenaSize = 8 * 1024;
byte tensorArena[tensorArenaSize] __attribute__((aligned(16)));
// array to map gesture index to a name
const char* GESTURES[] = {
"punch",
"flex"
};
#define NUM_GESTURES (sizeof(GESTURES) / sizeof(GESTURES[0]))
void setup() {
Serial.begin(9600);
while (!Serial);
// initialize the IMU
if (!IMU.begin()) {
Serial.println("Failed to initialize IMU!");
while (1);
}
// print out the samples rates of the IMUs
Serial.print("Accelerometer sample rate = ");
Serial.print(IMU.accelerationSampleRate());
Serial.println(" Hz");
Serial.print("Gyroscope sample rate = ");
Serial.print(IMU.gyroscopeSampleRate());
Serial.println(" Hz");
Serial.println();
// get the TFL representation of the model byte array
tflModel = tflite::GetModel(model);
if (tflModel->version() != TFLITE_SCHEMA_VERSION) {
Serial.println("Model schema mismatch!");
while (1);
}
// Create an interpreter to run the model
tflInterpreter = new tflite::MicroInterpreter(tflModel, tflOpsResolver, tensorArena, tensorArenaSize, &tflErrorReporter);
// Allocate memory for the model's input and output tensors
tflInterpreter->AllocateTensors();
// Get pointers for the model's input and output tensors
tflInputTensor = tflInterpreter->input(0);
tflOutputTensor = tflInterpreter->output(0);
}
void loop() {
float aX, aY, aZ, gX, gY, gZ;
// wait for significant motion
while (samplesRead == numSamples) {
if (IMU.accelerationAvailable()) {
// read the acceleration data
IMU.readAcceleration(aX, aY, aZ);
// sum up the absolutes
float aSum = fabs(aX) + fabs(aY) + fabs(aZ);
// check if it's above the threshold
if (aSum >= accelerationThreshold) {
// reset the sample read count
samplesRead = 0;
break;
}
}
}
// check if the all the required samples have been read since
// the last time the significant motion was detected
while (samplesRead < numSamples) {
// check if new acceleration AND gyroscope data is available
if (IMU.accelerationAvailable() && IMU.gyroscopeAvailable()) {
// read the acceleration and gyroscope data
IMU.readAcceleration(aX, aY, aZ);
IMU.readGyroscope(gX, gY, gZ);
// normalize the IMU data between 0 to 1 and store in the model's
// input tensor
tflInputTensor->data.f[samplesRead * 6 + 0] = (aX + 4.0) / 8.0;
tflInputTensor->data.f[samplesRead * 6 + 1] = (aY + 4.0) / 8.0;
tflInputTensor->data.f[samplesRead * 6 + 2] = (aZ + 4.0) / 8.0;
tflInputTensor->data.f[samplesRead * 6 + 3] = (gX + 2000.0) / 4000.0;
tflInputTensor->data.f[samplesRead * 6 + 4] = (gY + 2000.0) / 4000.0;
tflInputTensor->data.f[samplesRead * 6 + 5] = (gZ + 2000.0) / 4000.0;
samplesRead++;
if (samplesRead == numSamples) {
// Run inferencing
TfLiteStatus invokeStatus = tflInterpreter->Invoke();
if (invokeStatus != kTfLiteOk) {
Serial.println("Invoke failed!");
while (1);
return;
}
// Loop through the output tensor values from the model
for (int i = 0; i < NUM_GESTURES; i++) {
Serial.print(GESTURES[i]);
Serial.print(": ");
Serial.println(tflOutputTensor->data.f[i], 6);
}
Serial.println();
}
}
}
}
And here is the error for that code:
/Users/maxwelljohnson/Desktop/Gesture/Gesture.ino: In function 'void setup()':
/Users/maxwelljohnson/Desktop/Gesture/Gesture.ino:86:122: error: no matching function for call to 'tflite::MicroInterpreter::MicroInterpreter(const tflite::Model*&, tflite::AllOpsResolver&, byte [8192], const int&, tflite::MicroErrorReporter*)'
tflInterpreter = new tflite::MicroInterpreter(tflModel, tflOpsResolver, tensorArena, tensorArenaSize, &tflErrorReporter);
^
In file included from /Users/maxwelljohnson/Desktop/Gesture/Gesture.ino:23:0:
/Users/maxwelljohnson/Documents/Arduino/libraries/Arduino_TensorFlowLite/src/tensorflow/lite/micro/micro_interpreter.h:60:3: note: candidate: tflite::MicroInterpreter::MicroInterpreter(const tflite::Model*, const tflite::MicroOpResolver&, tflite::MicroAllocator*, tflite::MicroResourceVariables*, tflite::MicroProfilerInterface*)
MicroInterpreter(const Model* model, const MicroOpResolver& op_resolver,
^~~~~~~~~~~~~~~~
/Users/maxwelljohnson/Documents/Arduino/libraries/Arduino_TensorFlowLite/src/tensorflow/lite/micro/micro_interpreter.h:60:3: note: no known conversion for argument 3 from 'byte [8192] {aka unsigned char [8192]}' to 'tflite::MicroAllocator*'
/Users/maxwelljohnson/Documents/Arduino/libraries/Arduino_TensorFlowLite/src/tensorflow/lite/micro/micro_interpreter.h:50:3: note: candidate: tflite::MicroInterpreter::MicroInterpreter(const tflite::Model*, const tflite::MicroOpResolver&, uint8_t*, size_t, tflite::MicroResourceVariables*, tflite::MicroProfilerInterface*)
MicroInterpreter(const Model* model, const MicroOpResolver& op_resolver,
^~~~~~~~~~~~~~~~
/Users/maxwelljohnson/Documents/Arduino/libraries/Arduino_TensorFlowLite/src/tensorflow/lite/micro/micro_interpreter.h:50:3: note: no known conversion for argument 5 from 'tflite::MicroErrorReporter*' to 'tflite::MicroResourceVariables*'
/Users/maxwelljohnson/Documents/Arduino/libraries/Arduino_TensorFlowLite/src/tensorflow/lite/micro/micro_interpreter.h:40:7: note: candidate: constexpr tflite::MicroInterpreter::MicroInterpreter(const tflite::MicroInterpreter&)
class MicroInterpreter {
^~~~~~~~~~~~~~~~
/Users/maxwelljohnson/Documents/Arduino/libraries/Arduino_TensorFlowLite/src/tensorflow/lite/micro/micro_interpreter.h:40:7: note: candidate expects 1 argument, 5 provided
exit status 1
Compilation error: no matching function for call to 'tflite::MicroInterpreter::MicroInterpreter(const tflite::Model*&, tflite::AllOpsResolver&, byte [8192], const int&, tflite::MicroErrorReporter*)'
model.h (893.0 KB)
Arduino_TensorFlowLite-2.4.0-ALPHA.zip (1.5 MB)
OR if anyone has any updated tutorials or things I should explore regarding using ML or AI with arduinos for gesture recognition and could link those I'd also really appreciate that. Working through this has been a huge headache so any help is welcome!