Pages: [1]   Go Down
Author Topic: Best way to send/store data to a computer  (Read 1726 times)
0 Members and 1 Guest are viewing this topic.
New York, NY
Offline Offline
Newbie
*
Karma: 0
Posts: 41
Almighty Forum, help this newbie solve his problem
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Hello Everyone,

I'm trying to send some data readed from Arduino to a computer for further processing. I have 4 inputs (readed as float), a counter (int), and 4 char for separation -format--> counter;#;#;#;# . I checked the number of readings (on the serial monitor - 19200bps) for 1-second (using millis() to check the time) the result was 90reads/second. The main critiria I have is to read the data as fast as possible from the sensors for about 10 seconds, this time might increase (all data received at same frequency).

1) What would be the best way to receive the data and send it to the computer, storing the readed data into some array and after done with the reading beging to send the data to the computer (via serial)? Or send the data directly (via serial)?  Can there be any better method?

2) As Arduino only has 2KB of SRAM I do not think I will be able to store in the board. What will be faster, store on an EEPROM (24LC256) or send the data directly to the computer (via serial)? 

I will really appreciate your help. Thank you.
Logged

If you think you have it, you don't. If you think you don't have it, you still don't.

Seattle, WA USA
Online Online
Brattain Member
*****
Karma: 549
Posts: 46142
Seattle, WA USA
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
The main critiria I have is to read the data as fast as possible from the sensors for about 10 seconds, this time might increase (all data received at same frequency).
Ok, so why are you not sending the data as fast as possible?

Quote
on the serial monitor - 19200bps
The Arduino can send data at 115200 - that's 8 times as fast as you are sending it.

The limit on the speed that you can transmit the data is the baud rate, not how/when the Serial.print() command is called.
Logged

New York, NY
Offline Offline
Newbie
*
Karma: 0
Posts: 41
Almighty Forum, help this newbie solve his problem
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
The limit on the speed that you can transmit the data is the baud rate, not how/when the Serial.print() command is called.
Thanks PaulS, I learned something new today. I made the change, and I have more data transfered this way but sinks me even more into what would be the best way to get the data to the computer. 
Logged

If you think you have it, you don't. If you think you don't have it, you still don't.

Offline Offline
Newbie
*
Karma: 0
Posts: 7
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I have just started to use an arduino for the first time today, so take my opinion with a grain of salt. I'm sure that this kind of situation has happened before and there are easy ways to store data coming from a serial port. In my case I had to deal with a similar situation before and I ended up using python + pySerial. If you are familiar with any programming language, python will be easy to learn (even if it's just for manipulating strings) and offer you a lot of flexibility. This first lecture from google (

) should be enough to get you going, and this article (http://www.varesano.net/blog/fabio/serial%20rs232%20connections%20python) shows how simple it is to implement the RS232 link.
Logged

Georgina Ontario
Offline Offline
Sr. Member
****
Karma: 5
Posts: 437
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Your tests were meaningless. If you write to the console it eats time. DAMHIKT!

I use the Ethershield and UDP -- faster -- but there are other issues.

There is no best way.

All you can do is write or design the program or system, see how fast you have to send it and adopt the most "reasonable" way based on the speed required, your pocket book and your programming skill etc.

Some people use the Processing language. I use Delphi 2007, a Mega2560 and an Ethernet shield  -- and UDP packets. I know how to do it, and have the equipment. Serial probably wasn't fast enough -- but I did not take the time to write the code and prove it. Plus -- I want to be able to run the program from remote locations and controllers placed around the house.

Plus the same code should work on wireless if I get that for the Arduino...

If I log data to the console as well as send via UDP/Ethernet I lose half the packets or more.

Each to their own...
« Last Edit: April 03, 2011, 06:22:20 pm by WillR » Logged

Just another Hacker

New York, NY
Offline Offline
Newbie
*
Karma: 0
Posts: 41
Almighty Forum, help this newbie solve his problem
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

WillR, as Arduino is connected to the PC why would I need EthernetShield? I will take your advise about designing the systems and see what's faster but I have no idea on how to decide what will better. unfortunately, i'm not knowledgeable enough on micro-controllers or related stuff  smiley-cry.   
Logged

If you think you have it, you don't. If you think you don't have it, you still don't.

Offline Offline
Newbie
*
Karma: 0
Posts: 7
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

The reason you have been suggested to use the ethernet shield is that it would be much faster than USART.
As mentioned before, what you really need to ask yourself is how many samples do you really need per second. After that you could calculate on theory the data rate and the amount of "full packets" that you can send to the PC within your time limit (taking in consideration the time lost obtaining the data samples), but actually testing it would be simpler and give you real world results (keep the code fast and simple, you don't want to be wasting cycles). Once the data has arrive to the computer that is another issue, since collecting the data won't slow the system, but first check if USART is fast enough for you.
Logged

Seattle, WA USA
Online Online
Brattain Member
*****
Karma: 549
Posts: 46142
Seattle, WA USA
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
The reason you have been suggested to use the ethernet shield is that it would be much faster than USART.
Have you verified this? The Server and Client classes bear a close resemblance to the Serial class. I don't think that the ethernet approach will be any faster.
Logged

Offline Offline
Newbie
*
Karma: 0
Posts: 7
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
The reason you have been suggested to use the ethernet shield is that it would be much faster than USART.
Have you verified this? The Server and Client classes bear a close resemblance to the Serial class. I don't think that the ethernet approach will be any faster.
No, I haven't looked at the classes or the shield, my apologies. I just assumed that the atmega together with the ethernet chip in the shield would run fast enough to to offer a significant speed increment compared to USART. I should have not jump into conclusions.
Logged

New York, NY
Offline Offline
Newbie
*
Karma: 0
Posts: 41
Almighty Forum, help this newbie solve his problem
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Thanks a lot for your help and advices. I have been trying to test which is my best solution and so far serial seems to be better, I have the following to send:

Types        Bytes   # of each     KBytes      KBits
char          1               4            7.656       61.248
Int            2               1      
Float         4                4      
            
Readings =   348/Sec         
   
I also was to try to use the EEPROM but I have not been able to get my hands on it, I do know what else to order along with it as its not worth it to pay shipping for few little things that cost less than $2.
Logged

If you think you have it, you don't. If you think you don't have it, you still don't.

Georgina Ontario
Offline Offline
Sr. Member
****
Karma: 5
Posts: 437
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Thanks a lot for your help and advices. I have been trying to test which is my best solution and so far serial seems to be better, I have the following to send:

Types        Bytes   # of each     KBytes      KBits
char          1               4            7.656       61.248
Int            2               1      
Float         4                4      
            
Readings =   348/Sec         
   
I also was to try to use the EEPROM but I have not been able to get my hands on it, I do know what else to order along with it as its not worth it to pay shipping for few little things that cost less than $2.

Better than what? What did you compare to what?

On Ethernet UDP with small packets -- yes I have achieved that rate.
Logged

Just another Hacker

New York, NY
Offline Offline
Newbie
*
Karma: 0
Posts: 41
Almighty Forum, help this newbie solve his problem
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Thanks for commenting WillR. Now that I read it again, even I got confused. My apologies for the confusion. Serial is the only test that I have been able to make (short in peaces), instead of "and so far serial seems to be better", I meant to say "and so far serial seems to be good".

But I feel like the data rate at which I acquire may be too fast for serial. I hope this is not true.
 
By the way, I ended up ordering the EthernetShield, the EEPROM, and SD cardholder to try them. I totally ignored USB. I have been reading around on the SD/microSD and I found like 2 or 3 libraries. I like the idea of the SD/microSD as I will have larger storage but I have not found anything precise about the speed. Can anyone comment on this?
Logged

If you think you have it, you don't. If you think you don't have it, you still don't.

Pages: [1]   Go Up
Jump to: