Go Down

Topic: How to capture serial data via shell script for db (Read 24187 times) previous topic - next topic

displacedtexan

Mar 29, 2010, 10:47 pm Last Edit: Mar 29, 2010, 10:59 pm by displacedtexan Reason: 1
I am trying to capture serial data from an arduino sensor (humidity, temperature, etc) and have a shell script send to a server to add to a MySQL database.

Jeff Keyzer seems to do something similar here http://mightyohm.com/files/wifiradio/interface.sh:
Code: [Select]
while true      # loop forever
do
  inputline="" # clear input
 
  # Loop until we get a valid reading from AVR
  until inputline=$(echo $inputline | grep -e "^temp: ")
  do
     inputline=$(head -n 1 < /dev/ttyUSB0)
  done
  echo "$inputline"
done


When I run the above script on my linux machine and the below test code on my arduino, I don't get consistent results.  
Code: [Select]
int count = 0;
void setup()                    // run once, when the sketch starts
{
 Serial.begin(9600);           // set up Serial library at 9600 bps
}

void loop()                       // run over and over again
{
 Serial.print("temp: ");
 Serial.println(count);
 count++;
 delay (5000);
}


Anybody know what I am doing wrong?

I get stuff like :
temp: 5
temp: 7
temp: 9
temp: 2
temp: 27

etc

Thanks in advance!

udoklein

What do you see if you pick up the results with a serial monitor / terminal program? That is: does it look as if the arduino causes the issue or does it look like your script fails? I would guess the script causes the issue. For me it looks as if it swallows some stuff. No clue why. I would just not use a shell script for such a purpose. I would go for perl or python:

Code: [Select]

#!/usr/bin/python
# -*- coding: utf-8 -*-

import serial
import time
import sys

if len(sys.argv) == 3:
     ser = serial.Serial(sys.argv[1], sys.argv[2])
else:
     print "# Please specify a port and a baudrate, e.g. %s /dev/ttyUSB0 115200" % sys.argv[0]
     print "# using defaults /dev/ttyUSB0 9600"
     ser = serial.Serial("/dev/ttyUSB0", 9600)

while 1:
     sys.stdout.write(ser.readline())


Udo
Check out my experiments http://blog.blinkenlight.net

pluggy

#2
Mar 29, 2010, 11:54 pm Last Edit: Mar 30, 2010, 12:01 am by stephen_t Reason: 1
I use bash for getting stuff from my arduino, I have the arduino send the data in 4 separate lines much as you've done over and over and the essence of the bash script run once a minute is

Code: [Select]
cat /dev/ttyUSB0|head -n 8|tail -n 4 >> /root/arduino

It reads 8 lines and just uses the last 4 so that when it joins the converstation half way through a line the corrupted bit is ignored.

The bash script then continues by sucking the data out of the /root/arduino file created by the above.  (a ple of grep statements effectively)

Since the above appends, you'll need to rm the file when you're done or it just grows and grows.

displacedtexan

Udo, I am planning on connecting my Arduino to my Asus wl-520gu router's serial port.  The router only has 4mb of flash, about 2mb of which is taken up by OpenWRT.  I don't have the room to install php or perl so I was hoping to copy the data with a shell script over to my webserver for processing.

pluggy, the following code:
loop.sh:
Code: [Select]
while true
do
     cat /dev/ttyUSB0|head -n 8|tail -n 4 >> /home/arduino/serialdata
done

arduino sketch:
Code: [Select]
int count = 0;
void setup()                    // run once, when the sketch starts
{
 Serial.begin(9600);           // set up Serial library at 9600 bps
}

void loop()                       // run over and over again
{
 Serial.print("temp: ");
 Serial.println(count);
 count++;
 delay (1000);
}


results in:
Code: [Select]
temtemp: 5
temp:8
temptemp7

: 19temp21
24p: 25

after about 10 seconds...

Anybody else have any ideas how to get data from the serial port attached to an embedded linjx system with little space into a db on a webserver?

Thanks again for the suggestions so far!

John_Ryan

If you've got a wireless router / access point, get a WiFi Shield

http://www.asynclabs.com/

Serial to WiFi to router to Web .. easy.

displacedtexan

Quote
If you've got a wireless router / access point, get a WiFi Shield

http://www.asynclabs.com/

Serial to WiFi to router to Web .. easy.


I applaud those making WiFi shields but it doesn't make any economic sense, at least for me.  

For one thing, I already have the router, and for another the router was $32 after rebate.  So... for about half the cost of the shield, I get a router with a USB port, serial header, wifi, oh and by the way it runs embedded linux (using OpenWRT).

Plus, with the help of such a great community, hopefully I will learn a thing or two about electronics while I am a it ;)


John_Ryan

#6
Mar 31, 2010, 04:01 am Last Edit: Mar 31, 2010, 04:23 am by John_Ryan Reason: 1
Quote

I applaud those making WiFi shields but it doesn't make any economic sense, at least for me.  

For one thing, I already have the router, and for another the router was $32 after rebate.  So... for about half the cost of the shield, I get a router with a USB port, serial header, wifi, oh and by the way it runs embedded linux (using OpenWRT).

Plus, with the help of such a great community, hopefully I will learn a thing or two about electronics while I am a it Wink



I don't know what drugs you are on but "you" asked for suggestions, WiFi is the quickest and least expensive method of shifting data from a serial port, to Arduino, WiShield, then via that router of yours assuming it's wireless, to the internet.

Using a WiFi shield won't short change your electronics learning experience, it's not for newbies, but you might learn a valuable lesson that sometimes its better not to waste your time trying to reinvent the wheel, which depends on how much you value your own time.

Before the WiFi shield came along, I among many spent hundreds of hours over years trying various hardware hacks and multi-discipline software solutions for moving data from Arduino to the web. It was a labor of necessity because no other solutions existed, and at the end of the day, the WiFi shield solved a number of nagging problems that you will discover for yourself in good time.

Edit: I think there's an Ethernet shield which might be cheaper than WiFi, but then it's not wireless, but definitely worth a look.

Edit: Here are a few links that might be worth reading, I think the first one might be close to what your looking for.

http://www.geocities.jp/arduino_diecimila/wifi/a2p_ddwrt_en.html

http://blog.datasingularity.com/?p=50

http://itp.nyu.edu/~dbo3/SerialServer/SerialServer.html



displacedtexan

#7
Apr 01, 2010, 06:18 pm Last Edit: Apr 01, 2010, 06:29 pm by displacedtexan Reason: 1
Quote
I don't know what drugs you are on but "you" asked for suggestions, WiFi is the quickest and least expensive method of shifting data from a serial port, to Arduino, WiShield, then via that router of yours assuming it's wireless, to the internet.


I tried to word my response to prevent just such a post, but I guess I failed.  Regardless, I appreciate your suggestions.

I think I may have found a solution to my problem:

Code: [Select]


while true
do

temp=$(grep -m 1 temp /dev/ttyUSB0)

echo $temp  
echo $temp >> /home/user/serialdata
echo "wrote to serialdata"
scp /home/user/serialdata user@url.com:/home/user
echo "uploaded"
done


successfully does just what I am looking for after setting up ssh to login using public key authentication.  Once the string "temp" is seen by grep on the serial port, that line is assigned to the variable temp, stored locally, and then uploaded via scp.

Entering the data into MySql using a php script on the serverside should be trivial since the information will be there locally.

Thanks again for the help!

I found this site to be particularly helpful:
http://takenapart.com/?p=3

He's even using the same router!

Go Up