Smart Glasses for the Blind: Reading ESP32 Serial Port Information

Using Arduino Networking, Protocols, and Devices
1

Subject: Smart Glasses for the Blind: Reading ESP32CAM Serial Port Information

Hi geeks,

I am working on a project for smart glasses for the blind. The goal is to convert the visual information captured by the ESP32Cam camera into audible sound. I have already successfully trained my model on the Edge Impulse platform, and the results that come out of the serial port are pretty good.

My problem now is to read these information from the serial port and upload it to my smartphone for audible interpretation.

Do you have any ideas on how to do this?

Thank you for your help.

2

The methods in the serial input basics tutorial may be of interest.

Othrwise we need to know the form of the serial data (protocol, structure). Can you provide a sample of the data?

3

Hi @ben-becker,

I do not know enough about your application but on a first glance would suggest to take advantage of the RF capabilities of the ESP32.

If you don't like to dive into UDP or TCP/IP There is a simple Bluetooth serial lib, see here for further information https://randomnerdtutorials.com/esp32-bluetooth-classic-arduino-ide/

I fancy TCP via WiFi, there are lots of examples (that have to be adopted to your purpose), see
https://www.iotsharing.com/2017/05/tcp-udp-ip-with-esp32.html?m=1
or
https://techtutorialsx.com/2018/11/16/esp32-websocket-server-sending-binary-frame-to-client/

Bluetooth and TCP are obviously available at all state-of-the-art smartphones. Your (probably) more challenging task may be to write an application for Android or iOS...

For Android - unless you know better - a good start might be the MIT App Inventor.
See https://appinventor.mit.edu/
I don't know if the MIT App supports Bluetooth and/or TCP but it might be worth a try...

Good luck!

P.S.: MIT supports Bluetooth, just found this https://community.appinventor.mit.edu/t/bluetooth-hc-06-arduino-send-receive-send-text-file-multitouch-image/9518
:slight_smile:

And there's obviously also a TCP support
https://community.appinventor.mit.edu/t/tcp-ip-extension/7142

A lot to learn ... :wink:

1 Like
4

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.