General approach: my options REST, (g)RPC or MQTT? What's easily possible?

Dear fellow Makers,

I'm new to Arduino and IoT, therefore I would like to get your opinion on how to go about designing my project.
Basic functionality in a nutshell:

  1. Arduino takes a picture
  2. Send this picture to my Backend server (this will be what Im seeking your advice on)
  3. server evaluates the picture (any kind of machines learning algorithm...not relevant)
  4. server sends back response (e.g. "object is red" and "object is a ball" and "there is 1 object present")
  5. based on the response, Arduino does something

Now, I'm not quite sure how to go about step 2). I could do a basic POST REST request, where I wrap the image in the Body, and then tweak my REST response codes on server side and then consume that in my arduino (e.g. Ok("text"). Not very neat...

Another option is MQTT request/response pattern, however here I am worried that the a) size limitations of a MQTT package outbound cannot carry an image file b) not sure if arduino mqtt library implements the request/response pattern c) seems overly complex

Preferred option: If I was to do this anywhere else, I would go for gRPC / protobuf. Is gRPC possible to implement on Arduino? Is there HTTP2 support? Googling has not yielded much success unfortunately.

Again, Im brand new to Arduino and fairly new to programming, so forgive if I have made any errors in my post. Any advice is welcome!

EDIT1:
Forgot to mention, Im using a MKR1010 board with Wifi in case this is relevant.

Thanks!

Are you writing your own backend server software? If yes, then a raw TCP (or UDP) socket connection is the simplest. No need to deal with the extra overhead of HTTP or any standard protocol for that matter

Hi,

An esp32cam module may be the ideal option for this (although it sounds like you have the image capture already figured out?)
If you have a look at this sketch it shows how you can capture an image as a jpg then convert it to RGB data which you could then send to your server to analyse or even have the esp32cam it's self do some analysis on it (as they are surprisingly powerful).

BTW - You can send the entire RGB image with the command: client.write(rgb, ARRAY_LENGTH);

This looks like it might be the type of thing you are doing?

hzrnbgy:
Are you writing your own backend server software? If yes, then a raw TCP (or UDP) socket connection is the simplest. No need to deal with the extra overhead of HTTP or any standard protocol for that matter

Thanks for the suggestion!
I am writing my backend in ASP.NET. Hypothetically, it will be a response/request where potentially 1000 clients (arduinos, phones, browsers...) consume the "classification" service. What I need is therefore to implement a unique request/response architecture. I just looked into your suggestion, I could do something like (sockets - how to implement request response in a tcp connection? - Stack Overflow) to uniquely tie requests to responses, but then I feel like Im doing a protocol implementation myself. Additionally, I would need to create/figure out authentication (i.e. send tokens, configure server side to handle it - it's out of the box in the request pipeline for http, etc), figure out something like SSL/encryption of the packets both on client and server-side, so I feel I would be right back at what some of the existing protocols I mentioned already do, only with a shoddy personal implementation :slight_smile: Plus, I would need to implement this across all the other client device types.
For these reasons, I would still prefer gRPC for example.
What are your thoughts ? Am I overcomplicating and not seeing a simple workaround?

alanesq:
Hi,

An esp32cam module may be the ideal option for this (although it sounds like you have the image capture already figured out?)
If you have a look at this sketch it shows how you can capture an image as a jpg then convert it to RGB data which you could then send to your server to analyse or even have the esp32cam it's self do some analysis on it (as they are surprisingly powerful).
GitHub - alanesq/esp32cam-demo: esp32cam module demo / project starting point sketch using Arduino ide
BTW - You can send the entire RGB image with the command: client.write(rgb, ARRAY_LENGTH);

This looks like it might be the type of thing you are doing?
ESP32-CAM Image Classification using Machine Learning

Thanks for your reply! I looked into the links you provided. Yes, this indeed looks something very similar to what I want to do. As a matter of fact, this functionality is exactly what I need from the ESP32_Cam demo project you linked!!:

String payload = "{\"inputs\": [{ \"data\": {\"image\": {\"base64\": \"" + buffer + "\"}}}]}";

  buffer = "";
  // Uncomment this if you want to show the payload
  // Serial.println(payload);

  esp_camera_fb_return(fb);
  
  // Generic model
  String model_id = "aaa03c23b3724a16a56b629203edc62c";

  HTTPClient http;
  http.begin("https://api.clarifai.com/v2/models/" + model_id + "/outputs");
  http.addHeader("Content-Type", "application/json");     
  http.addHeader("Authorization", "Key your_key"); 
  int response_code = http.POST(payload);

In his (her? dont see a name) implementation HTTP POST is being used, which was one of the options I mentioned. This will work, however I see he only captures an int response code...I need significantly more than that. I could tweak the above of course, on that note can I just deserialise a JSON (i.e. payload) into a class instance (e.g. ClassifierResponse, with the relevant properties) ? That's how I would go about it in c#, not sure if I would need deconstruct the payload in a different way here?
Nevertheless, I would still prefer gRPC/HTTP2 though for the speed. Any chance it will ever be implemented for Arduino? Is it actually possible to take a native C++ implementation?
Sorry for the many follow-up questions :slight_smile: The link really made me think