Compilation issue with TensorFlow Lite for Microcontrollers in Arduino Web Editor

Development Tools IDE 1.x
1

Hello All,

I have been trying to compile a tensorflow lite for microcontroller example in the Arduino Web Editor, but I got the following compilation issue:

/home/builder/opt/libraries/latest/harvard_tinymlx_1_2_3_alpha/src/OV767X_TinyMLx.cpp: In member function 'int OV767X::begin(int, int, int, int)':

/home/builder/opt/libraries/latest/harvard_tinymlx_1_2_3_alpha/src/OV767X_TinyMLx.cpp:18:43: error: 'NRF_P0' was not declared in this scope

#define portInputRegister(P) ((P == 0) ? &NRF_P0->IN : &NRF_P1->IN)

^

/home/builder/opt/libraries/latest/harvard_tinymlx_1_2_3_alpha/src/OV767X_TinyMLx.cpp:138:16: note: in expansion of macro 'portInputRegister'

_vsyncPort = portInputRegister(digitalPinToPort(_vsyncPin));

^~~~~~~~~~~~~~~~~

/home/builder/opt/libraries/latest/harvard_tinymlx_1_2_3_alpha/src/OV767X_TinyMLx.cpp:18:57: error: 'NRF_P1' was not declared in this scope

#define portInputRegister(P) ((P == 0) ? &NRF_P0->IN : &NRF_P1->IN)

^

/home/builder/opt/libraries/latest/harvard_tinymlx_1_2_3_alpha/src/OV767X_TinyMLx.cpp:138:16: note: in expansion of macro 'portInputRegister'

_vsyncPort = portInputRegister(digitalPinToPort(_vsyncPin));

^~~~~~~~~~~~~~~~~

/home/builder/opt/libraries/latest/harvard_tinymlx_1_2_3_alpha/src/OV767X_TinyMLx.cpp: In member function 'void OV767X::readFrame(void*)':

/home/builder/opt/libraries/latest/harvard_tinymlx_1_2_3_alpha/src/OV767X_TinyMLx.cpp:244:1: error: 'NRF_GPIO_Type' was not declared in this scope

NRF_GPIO_Type * port;

^~~~~~~~~~~~~

/home/builder/opt/libraries/latest/harvard_tinymlx_1_2_3_alpha/src/OV767X_TinyMLx.cpp:244:1: note: suggested alternative: 'NVIC_Type'

NRF_GPIO_Type * port;

^~~~~~~~~~~~~

NVIC_Type

/home/builder/opt/libraries/latest/harvard_tinymlx_1_2_3_alpha/src/OV767X_TinyMLx.cpp:244:17: error: 'port' was not declared in this scope

NRF_GPIO_Type * port;

^~~~

/home/builder/opt/libraries/latest/harvard_tinymlx_1_2_3_alpha/src/OV767X_TinyMLx.cpp:244:17: note: suggested alternative: 'port_s'

NRF_GPIO_Type * port;

^~~~

port_s

/home/builder/opt/libraries/latest/harvard_tinymlx_1_2_3_alpha/src/OV767X_TinyMLx.cpp:246:10: error: 'nrf_gpio_pin_port_decode' was not declared in this scope

port = nrf_gpio_pin_port_decode(&ulPin);

^~~~~~~~~~~~~~~~~~~~~~~~

/home/builder/opt/libraries/latest/harvard_tinymlx_1_2_3_alpha/src/OV767X_TinyMLx.cpp: In member function 'void OV767X::beginXClk()':

/home/builder/opt/libraries/latest/harvard_tinymlx_1_2_3_alpha/src/OV767X_TinyMLx.cpp:369:3: error: 'NRF_I2S' was not declared in this scope

NRF_I2S->CONFIG.MCKEN = (I2S_CONFIG_MCKEN_MCKEN_ENABLE << I2S_CONFIG_MCKEN_MCKEN_Pos);

^~~~~~~

/home/builder/opt/libraries/latest/harvard_tinymlx_1_2_3_alpha/src/OV767X_TinyMLx.cpp:369:3: note: suggested alternative: 'NUM_I2CS'

NRF_I2S->CONFIG.MCKEN = (I2S_CONFIG_MCKEN_MCKEN_ENABLE << I2S_CONFIG_MCKEN_MCKEN_Pos);

^~~~~~~

NUM_I2CS

/home/builder/opt/libraries/latest/harvard_tinymlx_1_2_3_alpha/src/OV767X_TinyMLx.cpp:369:28: error: 'I2S_CONFIG_MCKEN_MCKEN_ENABLE' was not declared in this scope

NRF_I2S->CONFIG.MCKEN = (I2S_CONFIG_MCKEN_MCKEN_ENABLE << I2S_CONFIG_MCKEN_MCKEN_Pos);

^~~~~~~~~~~~~~~~~~~~~~~~~~~~~

/home/builder/opt/libraries/latest/harvard_tinymlx_1_2_3_alpha/src/OV767X_TinyMLx.cpp:369:61: error: 'I2S_CONFIG_MCKEN_MCKEN_Pos' was not declared in this scope

NRF_I2S->CONFIG.MCKEN = (I2S_CONFIG_MCKEN_MCKEN_ENABLE << I2S_CONFIG_MCKEN_MCKEN_Pos);

^~~~~~~~~~~~~~~~~~~~~~~~~~

/home/builder/opt/libraries/latest/harvard_tinymlx_1_2_3_alpha/src/OV767X_TinyMLx.cpp:370:29: error: 'I2S_CONFIG_MCKFREQ_MCKFREQ_32MDIV2' was not declared in this scope

NRF_I2S->CONFIG.MCKFREQ = I2S_CONFIG_MCKFREQ_MCKFREQ_32MDIV2 << I2S_CONFIG_MCKFREQ_MCKFREQ_Pos;

^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

/home/builder/opt/libraries/latest/harvard_tinymlx_1_2_3_alpha/src/OV767X_TinyMLx.cpp:370:68: error: 'I2S_CONFIG_MCKFREQ_MCKFREQ_Pos' was not declared in this scope

NRF_I2S->CONFIG.MCKFREQ = I2S_CONFIG_MCKFREQ_MCKFREQ_32MDIV2 << I2S_CONFIG_MCKFREQ_MCKFREQ_Pos;

^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

/home/builder/opt/libraries/latest/harvard_tinymlx_1_2_3_alpha/src/OV767X_TinyMLx.cpp:371:26: error: 'I2S_CONFIG_MODE_MODE_MASTER' was not declared in this scope

NRF_I2S->CONFIG.MODE = I2S_CONFIG_MODE_MODE_MASTER << I2S_CONFIG_MODE_MODE_Pos;

^~~~~~~~~~~~~~~~~~~~~~~~~~~

/home/builder/opt/libraries/latest/harvard_tinymlx_1_2_3_alpha/src/OV767X_TinyMLx.cpp:371:57: error: 'I2S_CONFIG_MODE_MODE_Pos' was not declared in this scope

NRF_I2S->CONFIG.MODE = I2S_CONFIG_MODE_MODE_MASTER << I2S_CONFIG_MODE_MODE_Pos;

^~~~~~~~~~~~~~~~~~~~~~~~

/home/builder/opt/libraries/latest/harvard_tinymlx_1_2_3_alpha/src/OV767X_TinyMLx.cpp:373:57: error: 'I2S_PSEL_MCK_PIN_Pos' was not declared in this scope

NRF_I2S->PSEL.MCK = (digitalPinToPinName(_xclkPin) << I2S_PSEL_MCK_PIN_Pos);

^~~~~~~~~~~~~~~~~~~~

/home/builder/opt/libraries/latest/harvard_tinymlx_1_2_3_alpha/src/OV767X_TinyMLx.cpp: In member function 'void OV767X::endXClk()':

/home/builder/opt/libraries/latest/harvard_tinymlx_1_2_3_alpha/src/OV767X_TinyMLx.cpp:381:3: error: 'NRF_I2S' was not declared in this scope

NRF_I2S->TASKS_STOP = 1;

^~~~~~~

/home/builder/opt/libraries/latest/harvard_tinymlx_1_2_3_alpha/src/OV767X_TinyMLx.cpp:381:3: note: suggested alternative: 'NUM_I2CS'

NRF_I2S->TASKS_STOP = 1;

^~~~~~~

NUM_I2CS

Error during build: exit status 1

The strange thing is that I have not used that file in my project, and my project was correctly compiled before. Here is the link to the source code: TinyML-Cookbook/09_classification.ino at main · PacktPublishing/TinyML-Cookbook · GitHub

Have you experimented with a similar issue before?

Many thanks for your help.
Gian Marco

2

Hi @gimmy987. Which Arduino board are you compiling for (i.e., which board do you have selected from the menu on the Arduino Web Editor toolbar)?

3

Hi @ptillisch , thanks for your reply! I used the Raspberry Pi Pico

4

@gimmy987 would it be possible for you to provide me with your model.h file?

I tried to investigate but I'm not at all knowledgeable with Jupyter so it was slow going trying to generate the model just so I can compile the sketch. I think I got it all set up correctly, but I was dumb enough to do that on my Windows machine and in the end it looks like the notebook is only usable on Linux. Using Linux is no barrier, but it's going to take a bunch more time to get Python 3.10 and all the dependencies installed on my Linux machine and I really don't have that much time to spend on the project.

I'll provide instructions for posting the file as an attachment in a reply here:

  1. Click the "Upload" icon (image) on the post composer toolbar:
    image
  2. Select the .txt file that contains the output.
  3. Click the Open button.
  4. Click the Reply button to publish the post.

Alternatively, instead of using the "Upload" icon on the post composer toolbar as described in the instructions above, you can simply drag and drop the .h file onto the post composer field to attach it.

5

Hi @ptillisch , thank you again for having a look at this issue. As requested, I attach the model.h to reproduce the issue in the Arduino Web Editor with Raspberry Pi Pico
model.h (115.5 KB)

6

By the way, I compiled with Arduino Nano 33 BLE sense, and the sketch has been compiled correctly. I suspect that the issue may come from the Raspberry Pi Pico and some missing macro guards in the file where I got the compilation failure.? :thinking:

7

I'll start with the fix, then follow with "the boring details":

The Fix

  1. Download the version of the "Arduino_TensorFlowLite" library that was created by Arduino from this link:
    https://downloads.arduino.cc/libraries/github.com/bcmi-labs/Arduino_TensorFlowLite-2.4.0-ALPHA.zip
  2. Wait for the download to finish.
  3. Select "Libraries" from the menu on the left side of the "Arduino Web Editor" window.
  4. Click the upward pointing arrow button ("Import") to the right side of the LIBRARY MANAGER button.
  5. If you get a popup about importing your sketchbook, click the IMPORT button.
  6. Select the "ZIP" file that contains the library.
  7. Click the Open button.
  8. Wait for "Arduino Web Editor" to display the notification that the library was successfully imported.
  9. Click the OK button.

You should now be able to compile your sketch for the Pico again without getting the errors.

Explanation

For some years, Arduino maintained a library named "Arduino_TensorFlowLite". This was "TensorFlow Lite Micro" packaged in a form that could be distributed as an Arduino library via the Arduino Library Manager.

The "TensorFlow Lite Micro" maintainers later created their own dedicated Arduino library project:

They requested the removal of the "Arduino_TensorFlowLite" library from Arduino Library Manager:

All the libraries of the Arduino Library Manager are pre-installed on Arduino Cloud. So you can compile Arduino sketches that depend on any of those libraries without having to first install the dependencies. Your sketch has a dependency on the TensorFlow Lite Micro library so it was using that library that used to be in Arduino Library Manager. When it was removed from library manager at the request of the "TensorFlow Lite Micro" maintainers, the dependency of your sketch was no longer available from the Arduino Cloud servers.

You might expect the removal of the dependency to result in an error like "TensorFlowLite.h No such file or directory", and that is what would happen if you tried to compile your sketch on your local computer after deleting the "Arduino_TensorFlowLite" library installation. But there are >5000 libraries in Arduino Library Manager, so when your sketch contains an #include directive for any common header filename, there will often be multiple libraries available that contain that header file. In this case, Arduino Cloud tries to guess the best library to use, and usually does a pretty good job at that. This was the case for the #include directives in your sketch. When the "Arduino_TensorFlowLite" library was available, Arduino Cloud correctly chose that library. But once that library was removed, it had to pick another library that happened to contain the specified files. You can see that happening in the verbose compilation output here:

Alternatives for TensorFlowLite.h: [harvard_tinymlx_1_2_3_alpha@1.2.3-Alpha]

ResolveLibrary(TensorFlowLite.h)

-> candidates: [harvard_tinymlx_1_2_3_alpha@1.2.3-Alpha]

So instead of using the , your sketch now uses the "Harvard_TinyMLx" library. This change in library dependency is the cause of the errors you experienced.

In theory, you should be able to use TensorFlow's new official Arduino library. After all, Arduino's library was only a mirror of the same "TensorFlow Lite Micro" project. I don't know anything about this "Harvard_TinyMLx" library, but it is also based on "TensorFlow Lite Micro", so theoretically it should also be usable. Unfortunately I did not have any success getting the sketch to compile with TensorFlow's library despite quite some trying. So out of desperation I dug up a download for Arduino's library.

Once you manually install that library in your Arduino Cloud account, it will once more be selected instead of the "Harvard_TinyMLx" library when compiling:

Multiple libraries were found for "TensorFlowLite.h"

Used: /mnt/create-efs/webide/a8/06/a806d47ed6d3f06510574743ac5887d3:pert_test/libraries_v2/Arduino_TensorFlowLite

Not used: /home/builder/opt/libraries/harvard_tinymlx_1_2_3_alpha

Please note that, following the removal request from TensorFlow, Arduino no longer maintains or supports the "Arduino_TensorFlowLite" library and there is no guarantee the download link I shared above will continue to work. So you might want to store a copy of that downloaded ZIP file in a safe place.

1 Like
8

That was extremely helpful! I managed to fix my problem, and now I can use Arduino Nano 33 BLE Sense and Raspberry Pi Pico with the latest TFLite Micro. I used the Google Arduino library project. To make it work, I had to remove some Arduino Nano 33 BLE Sense dependencies in the peripherals. Without this change, I could not compile TFLite micro for Raspberry Pi Pico. In particular, the following line does not allow to compile of the library for any Arduino boards except the Arduino Nano 33 BLE Sense:

I want to thank you again for your super help. Much appreciated!

9

I also ran into that. It is really unfortunate the library maintainers allowed such a degradation to the flexibility of the library. I reverted that change but then still ran into some other errors after that, which caused me to give up on getting the sketch working with that library.

I think you made the right choice by putting in the effort to get your sketch working with it though because TensorFlow's official library will likely be the most actively maintained. Hopefully they will start being more careful in the future.

You are welcome. I'm glad if I was able to be of assistance.

Regards,
Per

10

Hi gimmy, I need your help. Could you give me more details about how to fix the latest TFLite Micro for RPi Pico? I tried to remove the lines what you said but still don't work for me.

Thanks for your support.

11

Hi @ajsoto89 ! More than happy! Did you download and upload this version of TFLu in the Arduino IDE? https://downloads.arduino.cc/libraries/github.com/bcmi-labs/Arduino_TensorFlowLite-2.4.0-ALPHA.zip

I think, download that version is the simplest way to use TensorFlow Lite for Microcontrollers with the projects presented in the book. In fact, if you try to install the latest TFLu, you will encounter other issues related to the API change. By the way, I will update the README.md file on GitHub very soon to post all the instructions to fix this problem.

12

Hi gimmy987, thanks for your answer. Yes, I downloaded TFLu ALPHA version and it works without problems. Anyways, I'll be waiting for the update and meanwhile, I'll be working with the ALPHA version.

Thanks!

13

Thank you so much. I'm currently using Arduino Portenta H7 and trying to use visual recognition models in Arduino IDE, instead of OpenMV. In case it's helpful to someone else.

1 Like
14

Any update on this?

@esteesjose @ajsoto89 @gimmy987 @ptillisch

Does anyone have a more general method for using GitHub - tensorflow/tflite-micro-arduino-examples with other boards than just the Nano33BleSense.

I want to get it running for the Nicla boards and the Portenta.

1 Like
15

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.