next topic »

[on:]

Hello,

Arduino is supposed to transmit array of integer to a windows software I am going to write
array length = 250, array elements = integer
How can this be transmitted -serial- fast ?
Any trick?, for example transferring some bits can be transferred by one byte with bitshift operations.

Thank you

[Reply #1 on:]

Using Serial.print() causes the integers to be converted to strings, and the strings sent. There are advantages and disadvantages to this approach. The disadvantage is that the string is usually more bytes than the int. The advantage, though, is that delimiters are easy to add, because only a very small set of possible byte values are used.

Using Serial.write() to send the highByte() and lowByte() is faster. But, serial data transmission is NOT guaranteed. If you lose a byte, the low byte from one element will be combined with the high byte of another, resulting in completely corrupt data, with no way to detect that this has happened.

So, the first question that needs to be answered is why you are sending 500 bytes of data between the Arduino and the PC. The Arduino has not read 250 sensors, so where is this data coming from?

The second question is what is your definition of fast?

[Reply #2 on:]

If you not use the IDE serial monitor you can use higher baudrates e.g. 230400 or 345600. I even had success  with 500.000 baud .

Terminal program I used was putty, but other will work too. - see - http://arduino.cc/forum/index.php/topic,61381.msg444421.html#msg444421 -

fast enough?

[Reply #3 on:]

Data is collected from radio rx and sent after sync code, I always send the next arriving 250 slopes (ASK) then to be evaluated.
I now use 115200 baud rate and not use serial monitor anyway.
And I forget too mention : Now I am convertin g to string with println(strval) and yes,this is good and easy to parse.
it was exactly what I thought, too: needs more memory / is easy to parse (for example 9999 as string is each a byte because you use only the range 0..9) Thats why an idea would be to send integers up to 65536 in two bytes. (the data is always <25000). That should only need 2 bytes / transmission , insteas of e.g. 5bytes for "25000" as string.

[Reply #4 on:]

another way to send the data might be to send the relative data so instead of

600 600 600 610 611 612 610 630 625 ..

you send

600 0 0 10 1 1 -2 20 -5 ..    // 600 is initial value

if the numbers are close together you reduce the amount of data send quite a bit.
Note that you need some seperator between the numbers to keep them apart (you can use + and - for that)
600+0+0+10+1+1-2+20-5 .. 

Relative coding of numbers in text mode can save up to 50%+ but it depends quite a bit on the data stream

You can implement additions to this coding scheme to make it work better without being more difficult to parse (ok a bit)
+nnnn = relative number
-nnnn = relative number
*nnnn = absolute number (new reference point- the 600 in the sample above should be send as  *600

and even further
/nnnn = current index of the array -> quantity check
&nnnn = crc value so far -> quality and quantity check
#xxxx = comment

*600+0+0+10+1+1-2/7*630-5 .. 

By adding the option *nnnn you can determine for every next number if it is faster to send the relative value or the absolute value.

Finally if your numbers have runs of the same number you could consider runlength encoding
so instead of

600 600 600 600 600 600 600 600 600 600 600 600 601

send 12,600 601

If you want to include that in the coding scheme above something like
Rmmm*nnn  ==> a run of mmm times the number nnn

/0R3*600+10+1+1-2/7*630-5 ..

[Reply #5 on:]

If you are only sending numbers,  you could encode your decimal digits two to a byte ( 4 bits each)

[Reply #6 on:]

Hello,
now I have changed the baudrate and am still using serial.println (maybe last thing I am going to change)
But these data are not each similar, so I think + and - can even be more than the value itself (e.g. 1000;900;100 - it is 1000,minus100,minus800 - last data (minus800) is more to send  than 100)

Additionally, it is important to know *when* Arduino can send the next array's element. Otherwise it will always send and maybe too fast , thus the buffer will be overwritten and causes data loss!
So, my idea is the following:

Arduino pseudo code:

Code:

for i=0 to datalength-1
send(data[i]);
while (Serial.read()!=123); // wait until the data arrived

windows program pseudo code:

Code:

-loop-
readin(data);
if validdata then (
workwithdata(data);
send(123); // ok got that data, send next one
)

what do you think?

And , I already though about encoding decimal digits to use the byte range
I would use two bytes then, example (255,255) is the highest value then : 255 * 256^1 + 255 * 256^0 = 65,535 and this is enough - so each transmission then needs two bytes

[Reply #7 on:]

And , I already though about encoding decimal digits to use the byte range

How will you know if a byte is lost? How will you deal with that possibility?

[Reply #8 on:]

Ok, I have "modified" it like this:

addition to "Arduino pseudo code":

Code:

   int  val = 54123; //example to check  (data[x]);
    byte v1  = val / 256;
    byte v2  = val % 256;
    Serial.write(v1);
    Serial.write(v2);
    Serial.println("");

A byte cannot be lost , I think , because the sole problem is, that the transmission might be too fast, and so the buffer gets overwritten of new outgoing data. This wont happen because I do not send the next array's element until the last has really arrived.

I forgot that v1 or v2 or even both could be 13 or 10 which is interpreted as the delimiter for new data.
A solution would be to use [0..254] for any data and "255" for the delimiter. Then I could send , with two bytes :
(254,254) is the highest value then : 254* 255^1 + 254* 255^0 = 65,024

[Reply #9 on:]

A byte cannot be lost , I think

Well, I can tell you for a fact that they can.

A byte can contain any value in the range 0 to 255. The bytes sent by the println() method fall in that range. How will you distinguish between MSB, LSB, CR, LF and MSB, CR, LF, MSB, when the LSB gets lost? Or between MSB, LSB, CR, LF and MSB, LSB, CR, MSB when the LF gets lost?

Sending binary data requires special care, and that special care often makes it slower than sending ASCII data.

Of course, you could always say "Oh, that won't happen", and hope that it is true.

The simplest solution is to send the number of bytes up front, as ASCII data. Then, send the ints as bytes. Then, send some sort of "We be done" packet.

On the receiving end, compare the number of bytes received, if the "We be done" packet arrives intact, to the number expected, and discard the whole lot if there is a mismatch, or if the "We be done" packet is incomplete, because a byte was lost.

[Reply #10 on:]

How does SPI stack up against RS232 for speed? Maybe you could use a USB to SPI converter like http://www.microchip.com/stellent/idcplg?IdcService=SS_GET_PAGE&nodeId=1406&dDocName=en557594

[Reply #11 on:]

Ok, I have edited my reply

I forgot that v1 or v2 or even both could be 13 or 10 which is interpreted as the delimiter for new data.
A solution would be to use [0..254] for any data and "255" for the delimiter. Then I could send , with two bytes :
(254,254) is the highest value then : 254* 255^1 + 254* 255^0 = 65,024

And I always send 250 datas, so with this encoding 500 bytes .

[Reply #12 on:]

int  val = 54123; //example to check  (data

• );

should be   unsigned int   -