Hello Friends. It's been a long time. I was working on something today and thought I'd get your input.
I've been working with chatGPT a lot lately and I was wondering today about use cases involving Arduino. Obviously I could build code to drop API calls, but I'm sure that's done already but I wanted something that could actually use Arduino. So either giving GPT physical inputs and outputs or letting humans talk to arduino with natural language.
So here's a first attempt. GitHub - delta-G/ArduGPT: Letting ChatGPT control an Arduino board
The idea is to use chatGPT as an interpreter between human speaking and the commands that a home automation system might take. For this example my home automation setup is just an arduino UNO pretending to control three different things, lights, a fan, and an HVAC system.
Here's the example code for Arduino. It just grabs serial commands and sends back serial pretending that it turned things on or off. You can see where you'd put code to actually do those things instead of just saying it.
#define BUFFER_SIZE 10
void setup() {
Serial.begin(115200);
}
boolean receiving = false;
char c;
char buffer[BUFFER_SIZE];
int index;
void loop() {
if(Serial.available()){
char c = Serial.read();
if (c == '<'){
receiving = true;
index = 0;
buffer[index] = 0;
}
if(receiving){
buffer[index] = c;
buffer[++index] = 0;
if (index >= BUFFER_SIZE){
index--; // crude, but we will fix before we build anything that could overflow
}
if (c == '>'){
receiving = false;
processBuffer(buffer);
}
}
}
}
void processBuffer(char* buffer){
int len = strlen(buffer);
char command = 0;
if((buffer[0] == '<') && (buffer[len-1] == '>')){
command = buffer[1];
if(command == 'L'){
if(buffer[3] == '0'){
Serial.println("Turning Lights Off");
} else if(buffer[3] == '1'){
Serial.println("Turning Lights On");
}
else {
Serial.println("Unkown Argument");
}
}
else if(command == 'F'){
if(buffer[3] == '0'){
Serial.println("Turning Fan Off");
} else if(buffer[3] == '1'){
Serial.println("Turning Fan On");
}
else {
Serial.println("Unkown Argument");
}
}
else if(command == 'A'){
int temp = strtol(buffer+3, NULL, 10);
Serial.print("Setting HVAC to ");
Serial.println(temp);
}
else {
Serial.println("Unknown Command");
}
}
else {
Serial.println("Bad Command Format");
}
}Ordinarily the user would have to memorize these commands. With some complex string processing, we could take in a number of natural language like strings and extract the information we need, but with all the nuance of language it would likely still fail for a number of use cases.
Imagine that we need it to respond not only to "Lights Off" but also "Turn lights off", "Kill the lights", "cut the lights", "switch off the lights" and every other possible way of saying in english to turn off the light. It would be a lot of code and we'd still miss a few.
So here's where NLP comes in. In this super simple example, I am using GPT-3.5-turbo through langchain in python. If you're not familiar with this type of code, let me assure you that it's pretty simple. Most of the code is around building the llm chain and taking the inputs from the user and sending stuff to arduino. All of the magic happens in the prompt template.
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import serial, time
from langchain import OpenAI, PromptTemplate, LLMChain
from langchain.chat_models import ChatOpenAI
template = """
You are an LLM that is acting as a translator between human input and a home automation system.
You will be given a list of commands and arguments that each command can take.
You will decide based on the human input which command and arguments to pass to the output and format them appropriately.
Here is a list of commands you can use with descriptions of their arguments and output:
[
command: "L"
description: used to turn lights on or off
input: either of "0" for off or "1" for on
command: "F"
description: used to turn the fan on or off
input: either of "0" for off or "1" for on
command: "A"
description: used to set the thermostat for the HVAC system
input: the temperature to set
]
If you do not find a suitable command, then respond with "unable"
To format the command string, place a "<" symbol before the command, then the command, then a comma "," and then any arguments as a comma separated list. Finally include the ">" symbol.
Here are some examples of formatted command strings:
To turn the light on: <L,1>
To turn the fan off: <F,0>
To set the HVAC to 85 degrees <A,85>
Be sure to include the formatted command string in your response and do not include any < or > symbols anywhere else in the response.
Begin!
Human Input:
{input}
"""
class ArduGPT:
def __init__(self, device):
self.exit_flag = False
self.device = device
llm = ChatOpenAI(
model_name="gpt-3.5-turbo",
temperature=0
)
prompt = PromptTemplate(
input_variables=["input"],
template=template,
)
self.llm_chain = LLMChain(llm=llm, prompt=prompt)
return
def getUserInput(self):
print("\n*******************************\n\nHuman: ")
self.human_input = input("")
if self.human_input == "exit":
self.exit_flag = True
return
def printResponse(self):
if self.ai_output is not None:
print("\n*******************************\n\nAI: ")
print(self.ai_output)
if 'unable' not in self.ai_output:
self.device.write(bytes(self.ai_output, 'utf-8'))
time.sleep(0.1) ## sleep while we wait for response
returned = self.device.readline()
print("\n*******************************\n\nArduino: ")
print(returned)
return
def getResponse(self):
if self.human_input is not None:
self.ai_output = self.llm_chain.run(self.human_input)
return
def run(self):
self.getUserInput()
while not self.exit_flag:
self.getResponse()
self.printResponse()
self.getUserInput()
return
arduino = serial.Serial(port='/dev/ttyACM0', baudrate=115200, timeout=0.1)
ard = ArduGPT(arduino)
ard.run()Now when you run this code from the console you see this sort of output:
$ python ArduinoGptLight.py
*******************************
Human:
Please turn on the light
*******************************
AI:
Output:
<L,1>
*******************************
Arduino:
b'Turning Lights On\r\n'
*******************************
Human:
Please set the heat to 75
*******************************
AI:
Output:
<A,75>
*******************************
Arduino:
b'Setting HVAC to 75\r\n'
*******************************
Human:
cut off the fan
*******************************
AI:
Output:
<F,0>
*******************************
Arduino:
b'Turning Fan Off\r\n'
*******************************
Human:
kill the lights
*******************************
AI:
Response:
<L,0>
*******************************
Arduino:
b'Turning Lights Off\r\n'
*******************************
Human:
set the air to 68
*******************************
AI:
Output:
<A,68>
*******************************
Arduino:
b'Setting HVAC to 68\r\n'
*******************************
Human:
switch the lights on
*******************************
AI:
Output:
<L,1>
*******************************
Arduino:
b'Turning Lights On\r\n'
*******************************
Human:
exit
And you can see that the AI is accurately understanding what the human is asking and translating that into the command structure that is needed for the arduino code. You can certainly imagine that arduino code getting much more complex and controlling more things. I think this serves as just an example. I can also imagine that the prompt could be improved. That represents just a few edits to my original attempt. I'm learning that the prompt is where the magic is in these NLP projects.
Anyway, I wanted to leave this here in hopes that either someone would have some ideas to share for improvements or maybe someone finds the code useful. If it's in the wrong section please feel free to move it.
