ESP-32 ChatGPT robot project

Yeah I know I would have done it, but this week I had everyday a different last test for every subject, and I had to revise everything we've done in the last 3 yrs and then next monday we will have 20 minutes each person were you get a word and you have to connect it to each subject so I'll talk for my example about AI in different subjects like technology, geography, science, italian, english... I mean this is how it works here in Italy, that's why I had to pause

Ok, I understood.
Here's the code:


#include <WiFiClientSecure.h>
#include <ArduinoJson.h>
#include <ChatGPT.hpp>
#include <BluetoothSerial.h>

static const char *ssid = "Walid's Galaxy A53 5G";
static const char *password = "bngs5227";
WiFiClientSecure client;
ChatGPT<WiFiClientSecure> chat_gpt(&client, "v1", "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");
String result;
boolean newData = false;

BluetoothSerial SerialBT;

const byte numChars = 32;
char receivedChars[numChars];          //first type of data, which will be a text
char receivedCharForMoving;  //second type of data which Can be : F, B, R, L, S

const int MR1 = 12; //ESP32 pins (MR=Right Motor) (ML=Left Motor) (1=Forward) (2=Backward)
const int MR2 = 14; 
const int ML1 = 27;
const int ML2 = 26;

void setup() {
  //______________________________WiFi______________________________________________
  delay(1000);
  Serial.begin(115200);
  delay(1000);
  Serial.print("Connecting to WiFi network: ");
  Serial.print(ssid);
  Serial.println("'...");
  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED) {
    Serial.println("Connecting...");
    delay(500);
  }
  Serial.println("Connected to Wifi");
  // Ignore SSL certificate validation
  client.setInsecure();
  //______________________________Bluetooth___________________________________________
 SerialBT.begin("Galaxy A53 di Walid");

  //______________________________Motors______________________________________________
  pinMode(MR1, OUTPUT); 
  pinMode(MR2, OUTPUT);
  pinMode(ML1, OUTPUT);
  pinMode(ML2, OUTPUT);
}


void Forward(){
        //RIGHT MOTOR
      digitalWrite(MR1,HIGH);//MOVE FRONT
      digitalWrite(MR2,LOW); //MOVE BACK
      //LEFT MOTOR
      digitalWrite(ML1,LOW);//MOVE BACK
      digitalWrite(ML2,HIGH);//MOVE FRONT
}
void Backward(){
      digitalWrite(MR1,LOW);
      digitalWrite(MR2,HIGH);
      digitalWrite(ML1,HIGH);
      digitalWrite(ML2,LOW);
}
void Left(){
      digitalWrite(MR1,HIGH);
      digitalWrite(MR2,LOW);
      digitalWrite(ML1,HIGH);
      digitalWrite(ML2,LOW);
}
void Right(){
      digitalWrite(MR1,LOW);
      digitalWrite(MR2,HIGH);
      digitalWrite(ML1,LOW);
      digitalWrite(ML2,HIGH);
}
void Stop(){
      digitalWrite(MR1,LOW); 
      digitalWrite(MR2,LOW);
      digitalWrite(ML1,LOW); 
      digitalWrite(ML2,LOW); 
}

void loop() {
   receivedCharForMoving =(char)SerialBT.read();
   if (SerialBT.available()) {

    
    if(receivedCharForMoving == 'F')
    {
      Forward();
       
    }
    if(receivedCharForMoving == 'B')
    {
 
      Backward(); 
    }         
     if(receivedCharForMoving == 'L')
    {

      Left();
    }        
    if(receivedCharForMoving == 'R')
    {

      Right(); 
    }
    if(receivedCharForMoving == 'S')
    {
      Stop();
    }
  }

    recvWithEndMarker();
    showNewData();
}

void recvWithEndMarker() {
    static byte ndx = 0;
    char endMarker = '/';
    char rc;
    
    while (SerialBT.available() > 0 && newData == false) {
        rc = SerialBT.read();

        if (rc != endMarker) {
            receivedChars[ndx] = rc;
            ndx++;
            if (ndx >= numChars) {
                ndx = numChars - 1;
            }
        }
        else {
            receivedChars[ndx] = '\0'; // terminate the string
            ndx = 0;
            newData = true;
        }
    }
}

void showNewData() {
    if (newData == true) {
    Serial.println("[ChatGPT]");
    if (chat_gpt.simple_message("gpt-3.5-turbo-0301", "user", receivedChars, result)) {
         // Connect to the other phone via Bluetooth
        BluetoothSerial bt;
        bt.begin("Tab A 8.0");  // Set Bluetooth name of the other phone

        // Send the data to the other phone
        bt.println(result);

        // Disconnect from the other phone
         bt.end();
    } else {
      Serial.println("===ERROR===");
      Serial.println(result);
    }

        newData = false;
    }
}

I've worked on it for some hours, step by step like @StefanL38 and @GolamMostafa told me, and this should connect to both the BT devices, but I think(not sure about this) that at the begin it (the ESP32) will be able to receive the moving commands but after I send a question to ChatGPT it won't be able to receive the data anymore because the endmarker will be activated, or I am wrong?
Is everything else fine?
(P.S.) At line 17-18 there are the 2 chars for the 2 types of input:
receivedChars is the text that will be sent to ChatGPT
receivedCharForMoving is one only char where:
F is for Forward
B is for Backward
L is for Left
R is for Right
S is for Stop

Your code reads a single byte from bluetooth serial

   receivedCharForMoving =(char)SerialBT.read();

then checks if there is still something in the receive-buffer whatever this might be

   if (SerialBT.available()) {

after that
you do the repeated call to

    recvWithEndMarker();

This tries to receive all data
but in case not all characters belonging to the end-marker-message are in the receive-buffer yet
the while-loop inside receive with endmarkers is exited and then a byte will be read out of the receive-buffer into variable receivedCharForMoving
at the top of loop

   receivedCharForMoving =(char)SerialBT.read();

this means in that case a byte that belongs to the endmarker-message is picked away from the endmarker-message and in case this character is equal to one of the moving commands the robot will move.

You can avoid this if you have a special startmarker-character that will not be used in your endmarker-messages. Then you have to check for this special startmarker

Such small things for special cases make up 80% of the developping time to get the last 20% of functionality / reliability for the code.

Sometimes I have the impression that the "italian way" is to quickly reach a qualitylevel of works under normal circumstances.

best regards Stefan

Oh yeah, you're right. Should this be fine? The incoming text now has both a start and an EndMarker.


#include <WiFiClientSecure.h>
#include <ArduinoJson.h>
#include <ChatGPT.hpp>
#include <BluetoothSerial.h>

static const char *ssid = "Walid's Galaxy A53 5G";
static const char *password = "bngs5227";
WiFiClientSecure client;
ChatGPT<WiFiClientSecure> chat_gpt(&client, "v1", "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");
String result;
boolean newData = false;

BluetoothSerial SerialBT;

const byte numChars = 32;
char receivedChars[numChars];          //first type of data, which will be a text
char receivedCharForMoving;  //second type of data which Can be : F, B, R, L, S

const int MR1 = 12; //ESP32 pins (MR=Right Motor) (ML=Left Motor) (1=Forward) (2=Backward)
const int MR2 = 14; 
const int ML1 = 27;
const int ML2 = 26;

void setup() {
  //______________________________WiFi______________________________________________
  delay(1000);
  Serial.begin(115200);
  delay(1000);
  Serial.print("Connecting to WiFi network: ");
  Serial.print(ssid);
  Serial.println("'...");
  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED) {
    Serial.println("Connecting...");
    delay(500);
  }
  Serial.println("Connected to Wifi");
  // Ignore SSL certificate validation
  client.setInsecure();
  //______________________________Bluetooth___________________________________________
 SerialBT.begin("Galaxy A53 di Walid");
  
  //______________________________Motors______________________________________________
  pinMode(MR1, OUTPUT); 
  pinMode(MR2, OUTPUT);
  pinMode(ML1, OUTPUT);
  pinMode(ML2, OUTPUT);
}


void Forward(){
        //RIGHT MOTOR
      digitalWrite(MR1,HIGH);//MOVE FRONT
      digitalWrite(MR2,LOW); //MOVE BACK
      //LEFT MOTOR
      digitalWrite(ML1,LOW);//MOVE BACK
      digitalWrite(ML2,HIGH);//MOVE FRONT
}
void Backward(){
      digitalWrite(MR1,LOW);
      digitalWrite(MR2,HIGH);
      digitalWrite(ML1,HIGH);
      digitalWrite(ML2,LOW);
}
void Left(){
      digitalWrite(MR1,HIGH);
      digitalWrite(MR2,LOW);
      digitalWrite(ML1,HIGH);
      digitalWrite(ML2,LOW);
}
void Right(){
      digitalWrite(MR1,LOW);
      digitalWrite(MR2,HIGH);
      digitalWrite(ML1,LOW);
      digitalWrite(ML2,HIGH);
}
void Stop(){
      digitalWrite(MR1,LOW); 
      digitalWrite(MR2,LOW);
      digitalWrite(ML1,LOW); 
      digitalWrite(ML2,LOW); 
}

void loop() {

   if (SerialBT.available()) {
   receivedCharForMoving =(char)SerialBT.read();
    Serial.println(receivedCharForMoving);
    if(receivedCharForMoving == 'F')
    {   Serial.println("F");
      Forward();
       
    }
    if(receivedCharForMoving == 'B')
    {    Serial.println("B");
 
      Backward(); 
    }         
     if(receivedCharForMoving == 'L')
    {   Serial.println("L");

      Left();
    }        
    if(receivedCharForMoving == 'R')
    {   Serial.println("R");

      Right(); 
    }
    if(receivedCharForMoving == 'S')
    {  Serial.println("S");
      Stop();
    }
  }

    recvWithStartEndMarkers();
    showNewData();
}


void recvWithStartEndMarkers() {
    static boolean recvInProgress = false;
    static byte ndx = 0;
    char startMarker = '<';
    char endMarker = '>';
    char rc;
 
    while (SerialBT.available() > 0 && newData == false) {
        rc = SerialBT.read();

        if (recvInProgress == true) {
            if (rc != endMarker) {
                receivedChars[ndx] = rc;
                ndx++;
                if (ndx >= numChars) {
                    ndx = numChars - 1;
                }
            }
            else {
                receivedChars[ndx] = '\0'; // terminate the string
                recvInProgress = false;
                ndx = 0;
                newData = true;
            }
        }

        else if (rc == startMarker) {
            recvInProgress = true;
        }
    }
}

void showNewData() {
    if (newData == true) {
    Serial.println("[ChatGPT]");
    if (chat_gpt.simple_message("gpt-3.5-turbo-0301", "user", receivedChars, result)) {
         // Connect to the other phone via Bluetooth
        BluetoothSerial bt;
        bt.begin("Tab A 8.0");  // Set Bluetooth name of the other phone

        // Send the data to the other phone
        bt.println(result);

        // Disconnect from the other phone
         bt.end();
    } else {
      Serial.println("===ERROR===");
      Serial.println(result);
    }

        newData = false;
    }
}

you should do a compile-test before posting code
trying to be quick is slowing things down.

I've done it, it doesn't give me any error

when I was copying the code that you have posted there was this

    recvWithEndMarker();
    showNewData();
}

void recvWithStartEndMarkers() {

and this does not compile for very obvious reasons
As you can see in the history of your post you have edited it

You did not change anything else in your code which means the former described problem still exists.

You seem to be in a hurry for making it work. Or maybe you are hoping that more experienced users will do the modifications for you.
Not me.

I know I edited it, I simply modified it to don't do another post.

Isn't it because I have to do it before Monday?

Sorry but I'm not that type of person, because contrary to what you said, I prefer learning from the teacher than asking him the aswers during the test, because one day or another I will need those things and he won't be there and I'll get in trouble.
I mean, I'm trying to be kind, why are you being like this? I'm not gonna say rude cuz I don't know how your private life is, if you're having troubles with family, or friends or whoever, and I know how much is it frustrating when you're having bad times and people who don't know anything about it, keep saying you're a bad person

You have to find a way that both messages are received securely.

The messages without the start/end-marker
and
the messages with start/endmarker

The function recvWithStartEndMarkers() starts to readout from the buffer
only in case it comes across the start-marker flag receiveInProgress is set true

        else if (rc == startMarker) {
            recvInProgress = true;

this means if there are bytes in the receive-buffer that are not the startmarker the bytes get thrown away and are lost forever

if you send a non-start/endmarker command and your code is below

   if (SerialBT.available()) {
   receivedCharForMoving =(char)SerialBT.read();
    Serial.println(receivedCharForMoving);

the command will not be executed
the other way round if the startmarker of a start/end-marker-message is picked up at this line of code

void loop() {
   if (SerialBT.available()) {
   receivedCharForMoving =(char)SerialBT.read();
    Serial.println(receivedCharForMoving);

the below call of

    recvWithStartEndMarkers();

does not see the start-marker and will throw away the hole message
because of

    while (SerialBT.available() > 0 && newData == false) {
        rc = SerialBT.read();

So my suggestion is that if you want to keep the non-start/endmarker messages you have to use Serial.peek() instead of serial.read() because serial.peek() keeps the byte inside the receive-buffer

Ok, everything's clear, thank you.

It's the first time I use this so I think I am going to watch a couple tutorials before using it. I'll let you know if everything works, now that I've just finished developing the BT apps for the tablet and the phone.

Ok, so here's the code I'm using now:

#include <WiFiClientSecure.h>
#include <ArduinoJson.h>
#include <ChatGPT.hpp>
#include <BluetoothSerial.h>

static const char *ssid = "Walid's Galaxy A53 5G";
static const char *password = "bngs5227";
WiFiClientSecure client;
ChatGPT<WiFiClientSecure> chat_gpt(&client, "v1", "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");
String result;
boolean newData = false;

BluetoothSerial SerialBT;

const byte numChars = 32;
char receivedChars[numChars];          //first type of data, which will be a text
char receivedCharForMoving;  //second type of data which Can be : F, B, R, L, S

const int MR1 = 12; //ESP32 pins (MR=Right Motor) (ML=Left Motor) (1=Forward) (2=Backward)
const int MR2 = 14; 
const int ML1 = 27;
const int ML2 = 26;

void setup() {
  //______________________________WiFi______________________________________________
  delay(1000);
  Serial.begin(115200);
  delay(1000);
  Serial.print("Connecting to WiFi network: ");
  Serial.print(ssid);
  Serial.println("'...");
  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED) {
    Serial.println("Connecting...");
    delay(500);
  }
  Serial.println("Connected to Wifi");
  // Ignore SSL certificate validation
  client.setInsecure();
  //______________________________Bluetooth___________________________________________
 SerialBT.begin("ESP32 Walid");
  
  //______________________________Motors______________________________________________
  pinMode(MR1, OUTPUT); 
  pinMode(MR2, OUTPUT);
  pinMode(ML1, OUTPUT);
  pinMode(ML2, OUTPUT);
}


void Forward(){
        //RIGHT MOTOR
      digitalWrite(MR1,HIGH);//MOVE FRONT
      digitalWrite(MR2,LOW); //MOVE BACK
      //LEFT MOTOR
      digitalWrite(ML1,LOW);//MOVE BACK
      digitalWrite(ML2,HIGH);//MOVE FRONT
}
void Backward(){
      digitalWrite(MR1,LOW);
      digitalWrite(MR2,HIGH);
      digitalWrite(ML1,HIGH);
      digitalWrite(ML2,LOW);
}
void Left(){
      digitalWrite(MR1,HIGH);
      digitalWrite(MR2,LOW);
      digitalWrite(ML1,HIGH);
      digitalWrite(ML2,LOW);
}
void Right(){
      digitalWrite(MR1,LOW);
      digitalWrite(MR2,HIGH);
      digitalWrite(ML1,LOW);
      digitalWrite(ML2,HIGH);
}
void Stop(){
      digitalWrite(MR1,LOW); 
      digitalWrite(MR2,LOW);
      digitalWrite(ML1,LOW); 
      digitalWrite(ML2,LOW); 
}

void loop() {

   if (SerialBT.available()) {
   receivedCharForMoving =(char)SerialBT.peek();
    Serial.println(receivedCharForMoving);
    if(receivedCharForMoving == 'F')
    {   Serial.println("F");
      Forward();
       
    }
    if(receivedCharForMoving == 'B')
    {    Serial.println("B");
 
      Backward(); 
    }         
     if(receivedCharForMoving == 'L')
    {   Serial.println("L");

      Left();
    }        
    if(receivedCharForMoving == 'R')
    {   Serial.println("R");

      Right(); 
    }
    if(receivedCharForMoving == 'S')
    {  Serial.println("S");
      Stop();
    }
  }

    recvWithStartEndMarkers();
    showNewData();
}


void recvWithStartEndMarkers() {
    static boolean recvInProgress = false;
    static byte ndx = 0;
    char startMarker = '<';
    char endMarker = '>';
    char rc;
 
    while (SerialBT.available() > 0 && newData == false) {
        rc = SerialBT.read();

        if (recvInProgress == true) {
            if (rc != endMarker) {
                receivedChars[ndx] = rc;
                ndx++;
                if (ndx >= numChars) {
                    ndx = numChars - 1;
                }
            }
            else {
                receivedChars[ndx] = '\0'; // terminate the string
                recvInProgress = false;
                ndx = 0;
                newData = true;
            }
        }

        else if (rc == startMarker) {
            recvInProgress = true;
        }
    }
}

void showNewData() {
    if (newData == true) {
    Serial.println("[ChatGPT]");
    if (chat_gpt.simple_message("gpt-3.5-turbo-0301", "user", receivedChars, result)) {
         // Connect to the other phone via Bluetooth
        BluetoothSerial bt;
        bt.begin("ESP32 Walid 2nd");  // Set Bluetooth name of the other phone

        // Send the data to the other phone
        bt.println(result);
        Serial.println(result);
        // Disconnect from the other phone
         bt.end();
    } else {
      Serial.println("===ERROR===");
      Serial.println(result);
    }

        newData = false;
    }
}

I made some small changes for debugging, but it's a couple simple things on the usb serial port so it's nothing.
This is what I reached:

The device could connect to the phone and correctly receive both the "text" and the moving commands.
The device could succesfully connect to ChatGPT's API and receive the answer(displaying it on thw Serial Monitor),
but
Couldn't connect to the second device, because it begins the connection, sends the received data from the API even if it's not connected and instantly closes it, and then it doesn't restart the loop because the first SerialBT.begin() is in the void Setup() part.

I can easily move this to the loop but how do I make the ESP wait until a connection is estabilished to send "result" and then close it (the connection).

which device is the "second device" this device has a specific name and you should use this name to make everyting clear

How can a device that is

receive data??? and then send this data to where????

what is "it" in this case??

which loop??

dear walid:

you should clearly and exactly specify each thing with its specific name

Without clearly naming things You put your potential helpers in the sitatuion
that they have to look up your code re-read the code and your foggy description trying to distinguish what is what and might be still wrong.

Sure you have loops
void loop() does always loop.
This means you mean a different loop but which one ?

You are the one that wants help so help us to help you by posting a new description where you strictly avoid unspecific words like "it" , "this", "that" "loop"
but instead add words that are unique to the thing you mean.

best regards Stefan

2 Likes

By Serial Numbers, I have wanted to mean that you write down your tasks schedule of this project using 1., 2., ....... For example:
1. PC (NOT Tablet to allow Others to participate in your project)/works) will be connected with ESP32 for uploading sketch and debugging.

2. PC will be connected with another Arduino (say, UNO) for uploading sketch and debugging. UNO will be talking with your Android Phone via BT.

3. ESP32 and UNO will be talking via UART/SUART Port.
4. ESP32 will be connected with MEGA for talking using UART2 Port.
5. ESP32 will be talking with ChatGPT API using WiFi of Router.

Program Development Strategy:
1. Send textual command from Android Phone to ESP32 who will send it to MEGA who will send it to target Motor of the Robot.

2. Send textual request from Phone to ESP32 who will send it to ChatGPT who will send the response to ESP32 for onward transmission to MEGA and then to Robot.

I don't understand the role being played by the Tablet in this project?

Can you use the free version of ChatGPT in this project?

@StefanL38 Sorry for the bad description I'll try again. The problem is here:

After the ESP32 receives the string "result" from the API it instantly execute these commands (the ones I put up here). This means that the ESP begins the BT connection and instantly ends the BT connection so I can't get the string result on my tablet.
So I need to make the ESP wait until the string "result" gets totally send, to then close the connection.

I sincerely don't know how to explain it but I realised I was wrong... what I meant is that here

after this command when void loop() restarts, the esp wont restart the Bluetooth again.

Ok, understood.

  1. Esp32 connects to the internet
  2. Esp32 connects to my phone via Bluetooth
  3. My phone sends to the Esp32 two types of data: moving commands (F,B,L,R,S) or a text.
  4. If the Esp receives moving commands it sends them to my ln298n motor driver to make the robot move.
  5. If the Esp receives a text it sends it to ChatGPT API and gets the result,
    then
    (not working part)
  6. The esp momentally disconnects from the phone and creates a new BT connection with a tablet to send him the string result

I have an idea: if I directly send the string result to the phone, so that the Esp doesn't need to connect to the tablet, and then from the phone I send the string to the tablet do you think it will work?
P.S.

The tablet will be on the robot and when it receives the result string I will use a text to voice function so that it's like the robot is answering me.

No API's are not free, but OpenAI gives you 5dollars for free which are a lot because I've used the API at least 45 times and I only used 11 cents from the free 5 dollars

1 Like

Excellent description of task schedule.

Thank you.

I'm working on this, I've just finished the app's changes, I'm looking at the ESP's code.
I'll let you know

the moving commands must have a start-character so you can detect it is a moving-command
you should do this by using peek() because if this first character is not the start-charcter of amoving-command
then
it is the start-character of the chatGPT-answer that starts with a different start-character

The chatGPT-answer starts with a start-character and this this start-character is needed to make the start-end-character -receive-function work

Without the chatGPT-text-start-character the whole rest of this chatGPT-text will be thrown away
vanish, will be deleted, no longer accessable.
do you understand?

You are very quick in developping ideas. But you are greatly underestimating the time it needs to make such things work
You would have to create an app that will receive the chatGPT-answer on the phone and then send the chatGPT-answer to the tablet. By which app ? programmed in which language?
tested in which time?

How many weeks (not days) can you schedule your presentation into the future?
0,14 weeks = 1 day because on monday you have to present it.

If you have to present it on monday
add one thing at a time to make functional as much as you can todays evening and on sunday.
Do you understand? one thing at a time. Not all in parallel.

1 Like

Yes I know, I did it yesterday when you told me, and it's working perfectly.

I'm using MIT App inventor 2 and it's so easy that I developed both the phone's and the tablet's app in less than a hour and half, starting from zero.

And that's what I did, after you told me, one thing at a time, and now thanks to you and all the other people I finally finished. Everything is working perfectly. The tablet can receive the data correctly and with the text to speech function I can now hear the answer, so... nothing else to say. Thank you and see y'aa

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.