Serial.write zeros, speed or possibly another issue?

Using an uno, ethernet shield, and an rs232 shield. Goal is to request a URL with some data string (and some add’l strings later that just set digital pins) and then send the data string out using Serial.write. Before sending to Serial.write the data string needs to be altered from hex to decimal. What I am seeing is that unless I loop the Serial.write for each byte and add a delay of ~500ms, I don’t get the appropriate characters with long length strings (200 chars) or when multiple 0’s in a row are present in the string (it sends a couple correct characters and then it is random or sometimes all random). Unfortunately, this long delay for each write is causing issues on the receiving end (an LED sign). I would like to be able to send the data much quicker and not have to delay half a second for each byte so that the receiving end doesn’t have issues.

Here is the current program as it stands, any assistance would be appreciated. Thanks!

/*
 * http://$IP/?pinD=$somestring&pinL&pinH
 * 
 * pinD = data to send
 * pinH = horn
 * pinL = lights
 */
 
#if ARDUINO > 18
#include <SPI.h>
#endif

#include <Ethernet.h>
#include <TextFinder.h>

byte mac[] = { 0x90, 0xA4, 0xDA, 0x00, 0x6D, 0xF7 };
byte ip[] = { 192,168,2,202 };

Server server(80);

//conversion function for hex to decimal
uint8_t *convert(char *input)
{
  int len = strlen(input);
  
  if (len % 2 == 1) return 0;
  uint8_t *result = (uint8_t *)malloc(len / 2);
  int i;
  for (i = 0; i < len; i +=2)
  {
    int val;
    sscanf(input+i, "%2x", &val);
    result[i/2] = (uint8_t)val;
  }
  return result;
}

void setup()
{
  Serial.begin(9600);
  Ethernet.begin(mac, ip);
  server.begin();
  UCSR0C = UCSR0C | B00001000;     // we need two stop bits
  UCSR0C = UCSR0C & B11001111;    // make sure there is NO parity
}

void loop()
{
  Client client = server.available();
  if (client) {
    TextFinder finder(client);
    while (client.connected()) {      
      if (client.available()) {          
        int digitalRequests = 0;  // counters to show the number of pin change requests

        //used to grab the data string
        int gst = 0;                // total characters in string
        char buf[1024] = {};    // actual request
        char *pre = "=";          // pre string check
        char *post = "&";        // post string check
        //
        if( finder.find("GET /") ) {            
          // find tokens starting with "pin" and stop on the first blank line  
          while(finder.findUntil("pin", "\n\r")){  
            char type = client.read(); // D or A
            if( type == 'D') {                 
              //Serial.print("We got data :: ");
              gst = finder.getString(pre,post,buf,1024);  //get the data string
            }
            else if( type == 'L'){  //implement this later, for now just count if set
              //Serial.print("Lights!");
              digitalRequests++;
            }
            else if( type == 'H'){  //implement this later, for now just count if set
              //Serial.print("Horn!");
              digitalRequests++;
            }
            else {           
            }
          }
        }

        // standard header
        client.println("HTTP/1.1 200 OK");
        client.println("Content-Type: text/html");
        client.println();

        // output the number of pins handled by the request (debug)
        client.print(digitalRequests);
        client.print(" digital pin(s) written");
        client.println("
");
        client.println("
");
        client.print("In the sending buffer: ");
        client.print(buf);
        client.println("
");
        client.print("Converted for sending: ");
        
        uint8_t *converted = convert(buf);
        
        int length = strlen(buf)/2;

        // Looping for each byte right now.
        // Without looping and setting a huge delay,
        // it doesn't work.
        // Attempted changing the delay after
        // a 00 was written but that didn't help.
        // Both delays are the same here,
        // this works, but is way too slow.
        // If i lower the delays too much, they start 
        // writing junk, especially after several 00 00 00 sequences.
        for (int i = 0; i < length; i++)
        {
          if (converted[i] == 00) {
            Serial.write((byte)0x00);
            client.print("0");
            delay(500);
          } else {
            Serial.write(converted[i]);
            client.print(converted[i]);
            delay(500);
          }
        }

        free (converted);
        break;        
      } 
    }
    // give the web browser time to receive the data
    delay(1);
    client.stop();
  }
}

This sounds like an electrical problem. How have you wired it up?

These two do not go together.

Using an uno

#include <TextFinder.h>
        char buf[1024] = {};    // actual request
              gst = finder.getString(pre,post,buf,1024);  //get the data string

Just stacked shields and an rs232-to-usb cable to test (used two different cables to rule the cable out).

Not sure what you mean- the lines don't go together, due to the size? I have switched to using a mega and sized the buffer down to 512 if that is what you meant.

Still seeing the same issue, if I request the following URL for example: http://192.168.2.202/?pinD=4142430041&

I see the following output (requested four times and removed the .5s delay):
41 42 43 00 CC
41 42 43 00 DE
41 42 43 00 41
41 42 43 00 CC

I added a memset right under the buffer, thinking maybe something with initialization?
memset(buf,0,sizeof(buf));

Under some circumstances you return a NULL pointer:

//conversion function for hex to decimal
uint8_t *convert(char *input)
{
  int len = strlen(input);

  if (len % 2 == 1) return 0;
  ...
  return result;
}

But you go ahead and use it anyway:

        uint8_t *converted = convert(buf);

        int length = strlen(buf)/2;
...
        for (int i = 0; i < length; i++)
        {
          if (converted[i] == 00) {
            Serial.write((byte)0x00);
            client.print("0");
            delay(500);
          } 
          else {
            Serial.write(converted[i]);
            client.print(converted[i]);
            delay(500);
          }
        }

Why do you need to test for zero exactly?

I don't need to test for zero- I was testing using a shorter delay after non-0 and a longer for delay after 0 since it seemed like multiple sequences of zeros were causing issues.

I shouldn't get a null pointer b/c the length of the data string will always be even. I guess I should just add a simple test to avoid that.

due to the size?

Yes. The UNO has 2K of SRAM. Your array is using half of that. The libraries each need some. You are (were) most likely running out of memory, among other problems.

Ah okay, thank you. The IDE lets you compile something that is > the RAM on the board you have selected?

Anyhow, with the Mega I am getting the same issues as mentioned above. Any other thoughts?

The IDE lets you compile something that is > the RAM on the board you have selected?

Not larger than the Flash memory you have, no. But, it will happily compile code that uses every bit of SRAM and then some.

Interesting, good to know, thank you.

If anyone has any other ideas, feel free to let me know. Until then, will keep debugging.

Thanks!

df6_3:
Just stacked shields and an rs232-to-usb cable to test (used two different cables to rule the cable out).

How long is this cable? I'm wondering if noise or something is affecting it.

Tried two different make/model cables, both under 1ft in length.

df6_3:
Unfortunately, this long delay for each write is causing issues on the receiving end (an LED sign).

What is the part number/specs for this LED sign?

Hey Nick- I understand the question of specs but I will spare you of the details since the protocol doc. is in translated English from Chinese. If I send the bytes directly over a usb to serial cable to the sign, it works fine (using a ruby serial port implementation). But, the Serial.write (or some other function of my little arduino sketch) is not properly writing the bytes I am asking it to write. As mentioned before long strings and single and multiple 00 sequences are not properly being written. But, if I add a delay of 0.5s betw. writes of each byte, it outputs properly from the arduino. Could it be something with the RS232 shield? Schematics to it are here: http://www.cutedigi.com/pub/Arduino/arduino_RS232.pdf.

It would help if you had a logic analyzer and could see what was being sent to the sign.

It would help if you gave the sign's part number. Perhaps lets us judge if we understand the specs or not.

Just stacked shields and an rs232-to-usb cable to test (used two different cables to rule the cable out).

Is the sign USB? In what way is a rs232-to-usb being used to connect the shield to the sign?

What voltage levels are involved? Is the sign using +/- 12V?

Hi Nick,
Thank you for the continued replies.

I don't have a logic analyzer but what I mean by rs-232 to usb is an rs-232 cable from the rs-232 shield to the usb port on my machine. I am reading the bytes I am sending over rs-232 to debug since it isn't sending the correct bytes at this point in time. Once it is sending the correct bytes I will remove the rs-232 to usb cable and instead connect the rs-232 shield directly to the led sign using the provided cable from them (db9 to a proprietary 6p jack on the sign).

The sign is using +/- 12V which the rs-232 shield is supplying from my understanding of the schematic: http://www.cutedigi.com/pub/Arduino/arduino_RS232.pdf

Like I mentioned before, I would really like to be able to see/read the bytes I am sending properly first, debug whatever is going on here, and then connect it to the sign. If I write a sequence such as 41 42 43 00 41, it writes the first 4 sets correctly but the 5th is garbage. The problem gets worse with longer strings (200+ bytes) and with multiple 00 sequences. Something isn't quite right here but I can't figure out what exactly is the cause.

It sounds like from reading http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1257024767 and Private Site that I will need to swap pins 2 and 3 to talk to the sign eventually once I figure out why I can't write the bytes I am telling it to write.

df6_3:
I don't have a logic analyzer but what I mean by rs-232 to usb is an rs-232 cable from the rs-232 shield to the usb port on my machine. I am reading the bytes I am sending over rs-232 to debug since it isn't sending the correct bytes at this point in time.

How are you debugging exactly? Can you describe that process in more detail?

Sure! I am using GtkTerm, a terminal emulator on ubuntu. I am setting the baud rate to 9600, 2 stop bits, no parity (the settings for the led sign). I have also tried removing the 2 stop bits and explicit no parity setting from my sketch and config. of GtkTerm but I see the same results.

Try this:

void setup ()
{
 Serial.begin (9600); 
}

void loop ()
 {
   Serial.print ("ABC");
   for (byte i = 0; i < 20; i++)
     Serial.write ((byte) 0);
   Serial.print ("ABC");
   delay (100);
 }

If there is some bizarre hardware problem, this sketch will show it up. If this works properly your sketch isn’t doing what you think it is.