Receiving a 1536 byte string

Hi All,

I have a project for the MEGA which needs to collect a ~1536 byte serial "chunk" of ASCII information from one serial port and then println() the info on another serial port.
Considering that the size of this chunk of data is greater than the default RX_BUFFER_SIZE=128 defined in HardwareSerial.cpp, it seems that I need to change this value, however i'm also seeing some uint_8t definitions in this file. Is setting an RX_BUFFER_SIZE=1536 going to be compatible with these uint_8t lines, or do i need to hack these lines and change them to uint_16t's?
Has anyone done this before in 0022?
Will I also run into problems trying to Serial.println() large strings?
Otherwise, any guidance would be greatly appreciated.
Cheers.

Why not just break the packet down into bits that will fit into the receive buffer?

Hi,
the reason I don’t split the incoming data is because I am communicating with a closed source device which returns this large chunk of data. I am able to request the chunk to be in binary, reducing the size of the dump to 512 bytes, but I still don’t think this can help me if the rx buffer is hard coded to 128.
Can anyone provide me info on how to modify HardwareSerial.cpp to allow me to recieve the string, or provide another avenue to capture the data?
Thanks

Can you control the baud rate on the input or output devices? If so, ensure that the baud rate going out is faster than coming in & your buffer size is somewhat irrelevant. This assumes that you will not be transforming the data as it passes through the Arduino (in which case, why is is there?) or that you don’t need the entire packet in memory to work the transform. If you do, perhaps better to use up some precious ram on the mega and use the 512 byte packet size and a byte array to put it in. What are you trying to do?

Thanks for your advice wildbill,
I am very new to C and hardware for that matter, so i'm not sure how reducing the baud rate will help me unless i have time during one 1536 byte "dump" to fill up my buffer, flush it, fill up my buffer, flush it ,fill up my buffer, flush it etc until I have stored all the values into another array. Is this what you mean by the buffer size becomes irrelevant?

To provide some more background on what I'd like to do:
The device is a linear array which collects photon charge for a user-defined (via a serial command) time period, referred to as the "integration time".
The linear array returns 256 x 16bit numbers (ie 256 different values ranging from 0 to 2^16) when commanded to do so.
I need the Arduino to check for saturated pixels (ie if any of the 256 numbers are close to 2^16), and if required, i need to alter the next user-defined time period to stop saturation.

So I would be wanting the code to not only stream the output linear array data from one serial to another, but also search for the maximum value of the 256 numbers.
There are a few other serial devices (GPS, Compass) which I also cycle through sequentially and I want these to appear as lines on the serial output serial / terminal
i.e.
$GPPGA, blah, blah (from the GPS)
$C180.0,103,110, blah (from the Compass)
$10002,65536,0,1203,24310,3523, etc.... (256 comma-separated numbers from the Linear Array)
and so on...

I am very new to all this, so any advice is greatly appreciated!

Is this what you mean by the buffer size becomes irrelevant?

No. If you can shuffle data out faster than it comes in, the 128 byte buffer will never fill up. The speed that data comes in is limited by the incoming baud rate. The rate that it goes out is limited by the outgoing baud rate AND the time you spend manipulating the data.

I need the Arduino to check for saturated pixels (ie if any of the 256 numbers are close to 2^16), and if required, i need to alter the next user-defined time period to stop saturation.

This does not explain why you also need to forward the data.

Thanks Paul, I have some more direction for tinkering now.
Also, I neglected to mention that I want the serial outputs from the gps, compass and linear array to be available to be read by a pc for subsequent analysis + storage.
Cheers

Looks feasible. Serial communications are slow compared to Arduino’s speed. It should be possible to check for saturation on the fly as you read the data - no need to buffer it. GPS tends to run at 4800 baud & emit once a second - nice and slow, hopefully the compass is similar. Crank up the baud rate on the communication to the PC and you should have ample time to service your serial ports, process the data and pass it on, as well as adjusting the integration time when required.