String Nightmares and More

Hi guys,

I’m currently working on a ground station transmitter controller for a satellite that I helped build over the summer. It’s getting ready to launch October 27th, so I’m in a bit of a hurry to get the final touches on the software end of things to control the transmitter. Here’s what I’ve run into tonight:

  • char pointers (and pointers in general) can be quite dangerous with Arduinos. I have succeeded in wiping the bootloader on two Diecimilas tonight before I “discovered” the issue. All I really found out was that using “char*” wasn’t a good idea.
  • String objects seem very glitchy. I was not able to use String arrays to store a lot of text and then print it out later. The arrays seemed to overwrite themselves or something. I’m not quite sure what was going on to be honest.
  • The same code loaded onto a working Diecimila wouldn’t run, whereas it would run just fine on my new Uno. What’s up with that?!

Here’s the code:

//AubieSat-1 Ground Station Transmitter Controller/Interface
//Rev. 0 (10/15/2011)

/* Pin Definitions */
int IPA_Switch = 3;
int HPA_Switch = 4;
int Coax_Switch = 5;
int Spare_Switch = 6;
int Encoder_TE = 7;
int Encoder_AD11 = 8;
int Encoder_AD10 = 9;
int Encoder_AD9 = 10;
int Encoder_AD8 = 11;
int Melexis_FSKD = 12;
int Melexis_ENTX = 13;

/* Commands */
int numberOfCommands = 11;

char commands[][15] = 
{
  "txtime        ",
  "transmit      ",
  "hpaenable     ",
  "encoderenable ",
  "carrierenable ",
  "moduleenable  ",
  "antennaswitch ",
  "spareenable   ",
  "fskdata       ",
  "command       ",
  "help          "
};

char shortcuts[] =
{
  'x',
  't',
  'h',
  'e',
  'i',
  'm',
  'a',
  's',
  'f',
  'c',
  '?'
}; 

char arguments[][20] =
{
  "<0 to 10000>       ",
  "[hexadecimal byte] ",
  "<0 or 1>           ",
  "<0 or 1>           ",
  "<0 or 1>           ",
  "<0 or 1>           ",
  "<0 or 1>           ",
  "<0 or 1>           ",
  "<0 or 1>           ",
  "<hexadecimal byte> ",
  "[]                 "
}; 

char description[][36] =
{
  "sets the transmit time in ms       ",
  "transmits the command              ",
  "enables/disables the HPA           ",
  "enables/disables the encoder       ",
  "enables/disables the carrier signal",
  "enables/disables the module        ",
  "switches the coaxial relay         ",
  "enables/disables the spare relay   ",
  "sets the FSK data                  ",
  "sets the command data              ",
  "displays the valid commands        "
}; 



/* Other variables */
byte lastCommand = 0;
int currentTransmitTime = 1000;

void setup()
{
  Serial.begin(19200);
  Serial.println("Welcome to the AubieSat-1 Ground Station Transmitter Command Prompt");
  Serial.println("Rev. 0 (10/15/2011)");
  Serial.println("~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~");
  Serial.println("Warning: This unit controls a 100W amplifier. Do not operate this device");
  Serial.println("without a proper 50 ohm load on the output. Also, make sure the coaxial");
  Serial.println("relay is switched to the proper position when transmitting. Failure to do");
  Serial.println("so will result in serious injury or death...to the equipment. Operating");
  Serial.println("this transmitter from console assumes you know what you are doing. If you");
  Serial.println("are unsure of what you are doing and/or you are not holding a valid amateur");
  Serial.println("radio license, you should close this terminal window now.\n");
  

  help();
  
  for (int i = IPA_Switch; i <= Melexis_ENTX; i++)
  {
    pinMode(i, OUTPUT);
  }
}

void help()
{
  Serial.println("Available commands are:");
  //Serial.println("txtime         (x)   <0 to 10000>        sets the transmit time in ms       ");
  Serial.println("COMMAND     (SHORTCUT)   ARGUMENT                DESCRIPTION");
  for (int i = 0; i < 11; i++)
  {
    Serial.print(commands[i]);// + "(" + shortcuts[i] + ") " + arguments[i]);// + description[i]);
    Serial.print(" (");
    Serial.print(shortcuts[i]);
    Serial.print(")   ");
    Serial.print(arguments[i]);
    Serial.print(" ");
    Serial.println(description[i]);
  }
  Serial.println("\nArguments in [] indicate that the argument is optional and that the command");
  Serial.println("will be executed with or without an argument. Arguments in <> indicate that");
  Serial.println("the command will echo the current configuration without executing the command.");
}

void loop()
{
  //Serial.println("This is a test.");
  //delay(500);
}

I would have preferred to use String objects wherever possible, but most of the time, the text output was composed of parts of other strings elsewhere in the program, even string literals. Sounds like a mis-malloc() to me, but I’m not positive.

I guess my biggest questions are these:

Why won’t this code run on my working Diecimila, but it will run fine on my Uno?
Why won’t this code work well with String objects, or will it but I’m doing something wrong?
Why can’t I concatenate a string literal (const char) with a character array (char)? Is it not common practice to do such a concatenation in a println() statement?
Why can I make a 2D char array (3D char array) with string literals in braces, but I can’t make a 2D char array from char arrays in braces? Like this:

char c[][2] = {{'a', 'b'}, {'c', 'd'}}; //this works

vs.

char a[] = {'a', 'b'};
char b[] = {'c', 'd'};
char c[][2] = {a, b}; //this doesn't...

Thank you very much!

Yes, you need to be careful using char* pointers, in particular to make sure that you never use them to access data outside the bounds of the char array that they point to. However, I can't recommend the use of the String class on the Arduino, because of its high and relatively unpredicatable memory usage, and the memory fragmentation that can occur when you use it. The Arduino has very limited RAM and will just crash when you run out.

So my advice is to stick with char* but be very careful when accessing (in particular, writing) data through a char*.

Why won't this code run on my working Diecimila, but it will run fine on my Uno?

The Diecimila has half the amount of SRAM. You have a large amount of constant string data occupying what little SRAM the arduino has available. Without counting it up, I suspect there's just too much text there to fit. Consider using PROGMEM for them and look in the forums for freemem or freememory functions to confirm how much memory you have left.

All I really found out was that using "char*" wasn't a good idea.

Wrong. It is misusing them that is not a good idea.

String objects seem very glitchy. I was not able to use String arrays to store a lot of text and then print it out later. The arrays seemed to overwrite themselves or something. I'm not quite sure what was going on to be honest.

String objects and large amounts of text and embedded systems do not go together. To see why, look at WString.cpp and the implementation of the += operator.

The same code loaded onto a working Diecimila wouldn't run, whereas it would run just fine on my new Uno. What's up with that?!

What chip do you have on the Diecimila? If it is a 168, it's no surprise the code won't work - not enough memory.

I would have preferred to use String objects wherever possible, but most of the time, the text output was composed of parts of other strings elsewhere in the program, even string literals.

"I would have preferred to be lazy and use Strings..."

Sounds like a mis-malloc() to me, but I'm not positive.

Sounds like you are running out of memory, to me.

int numberOfCommands = 11;

char commands[][15] = 
{
};

There went 165 of the 2048 bytes of SRAM...

char arguments[][20] =
{
};

There went another 320...

char description[][36] =
{
};

And another 1260.

That's 1,745 of 2,048 before you even get to the functions.

You need to look into PROGMEM for all these constant strings.

char c[][2] = {a, b}; //this doesn't...

You can assign values to an array at declaration time. You can not assign the value of an array to another array - the assignment does not copy all the elements of a to c.

You should look at using pointers, and arrays of pointers.

Of course, given your first statement, that probably isn't a good idea.

May I offer a couple observations?

  1. String constants live in RAM. It looks like you're using a lot of it this way, by '328 standards. It may explain the flaky behavior. A forum search will find functions to tell you how much free ram you have at runtime. It's worth checking. As an alternative, you can declare strings in Flash memory and print from there; another forum search.

  2. Shameless Bitlash plug: You appear to be writing a command interpreter. It may be worth considering using the Bitlash interpreter library as a starting point for your ground station controller.

Bitlash (http://bitlash.net) is an interpreter for Arduino which is easy to extend to include your hardware control functions - you can expose your stuff as "user functions" in the Bitlash script language, and call them like any other Bitlash function.

Bitlash handles all the parsing and console IO, and calls your functions with the arguments the user passed in the console command. You can store Bitlash functions in EEPROM - may not be a requirement for your application but it sure comes in handy.

Source and documentation at http://bitlash.net -- good luck with your project, it looks very interesting.

Best,

-br http://bitlash.net

Trying your code with these lines alone commented out saved 888 bytes of RAM (out of your 1024 you had):

  Serial.println("Welcome to the AubieSat-1 Ground Station Transmitter Command Prompt");
  Serial.println("Rev. 0 (10/15/2011)");
  Serial.println("~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~");
  Serial.println("Warning: This unit controls a 100W amplifier. Do not operate this device");
  Serial.println("without a proper 50 ohm load on the output. Also, make sure the coaxial");
  Serial.println("relay is switched to the proper position when transmitting. Failure to do");
  Serial.println("so will result in serious injury or death...to the equipment. Operating");
  Serial.println("this transmitter from console assumes you know what you are doing. If you");
  Serial.println("are unsure of what you are doing and/or you are not holding a valid amateur");
  Serial.println("radio license, you should close this terminal window now.\n");
  
  Serial.println("\nArguments in [] indicate that the argument is optional and that the command");
  Serial.println("will be executed with or without an argument. Arguments in <> indicate that");
  Serial.println("the command will echo the current configuration without executing the command.");

That stuff, if you need all the wordiness, should certainly be in PROGMEM.

Thanks for the tips, guys! Here’s what it’s looking like now:

//AubieSat-1 Ground Station Transmitter Controller/Interface
//Rev. 0 (10/15/2011)

#include<avr/pgmspace.h>

/* Pin Definitions */
#define IPA_Switch 3
#define HPA_Switch 4
#define Coax_Switch 5
#define Spare_Switch 6
#define Encoder_TE 7
#define Encoder_AD11 8
#define Encoder_AD10 9
#define Encoder_AD9 10
#define Encoder_AD8 11
#define Melexis_FSKD 12
#define Melexis_ENTX 13

/* Commands */
#define numberOfCommands 11

//command listing
prog_char string_0[] PROGMEM = "txtime         (x)   <decimal int>       sets the transmit time in ms       ";
prog_char string_1[] PROGMEM = "transmit       (t)   [decimal byte]      transmits the command              ";
prog_char string_2[] PROGMEM = "hpaenable      (h)   <0 or 1>            enables/disables the HPA           ";
prog_char string_3[] PROGMEM = "encoderenable  (e)   <0 or 1>            enables/disables the encoder       ";
prog_char string_4[] PROGMEM = "carrierenable  (i)   <0 or 1>            enables/disables the carrier signal";
prog_char string_5[] PROGMEM = "moduleenable   (m)   <0 or 1>            enables/disables the module        ";
prog_char string_6[] PROGMEM = "antennaswitch  (a)   <0 or 1>            switches the coaxial relay         ";
prog_char string_7[] PROGMEM = "spareenable    (s)   <0 or 1>            enables/disables the spare relay   ";
prog_char string_8[] PROGMEM = "fskdata        (f)   <0 or 1>            sets the FSK data                  ";
prog_char string_9[] PROGMEM = "command        (c)   <decimal byte>      sets the command data              ";
prog_char string_10[] PROGMEM = "help           (?)   []                  displays the valid commands        ";

prog_char messageOfTheDay[] PROGMEM = //MOTD
"Welcome to the AubieSat-1 Ground Station Transmitter Command Prompt\n"
"Rev. 0 (10/15/2011)\n"
"~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n"
"Type '?' to see the help page at any time.\n\n"
"Warning: This unit controls a 100W amplifier. Do not operate this device\n"
"without a proper 50 ohm load on the output. Also, make sure the coaxial\n"
"relay is switched to the proper position when transmitting. Failure to do\n"
"so will result in serious injury or death...to the equipment. Operating\n"
"this transmitter from console assumes you know what you are doing. If you\n"
"are unsure of what you are doing and/or you are not holding a valid amateur\n"
"radio license, you should close this terminal window now.\n";

prog_char referenceInfo[] PROGMEM = //Command reference info
"Arguments in [] indicate that the argument is optional and that the command\n"
"will be executed with or without an argument. Arguments in <> indicate that\n"
"the command will echo the current configuration without executing the command.\n";


PROGMEM const char *commands[] =
{
  string_0,
  string_1,
  string_2,
  string_3,
  string_4,
  string_5,
  string_6,
  string_7,
  string_8,
  string_9,
  string_10
};

char commandBuffer[77];

/* Other variables */
byte lastCommand = 0;
int currentTransmitTime = 1000;

void setup()
{
  Serial.begin(57600);
  
  motd();
  help();
  
  for (int i = IPA_Switch; i <= Melexis_ENTX; i++)
  {
    pinMode(i, OUTPUT);
  }
}

void motd()
{
  for (int i = 0; i < sizeof(messageOfTheDay); i++)
  {
    Serial.print(pgm_read_byte(messageOfTheDay + i));
  }
  Serial.println();
}

void help()
{
  Serial.println("Available commands are:");
  Serial.println("COMMAND     (SHORTCUT)   ARGUMENT                DESCRIPTION");
  for (int i = 0; i < numberOfCommands; i++)
  {
    strcpy_P(commandBuffer, (char*)pgm_read_word(&(commands[i])));
    Serial.println(commandBuffer);
  }
  Serial.println();
  for (int i = 0; i < sizeof(referenceInfo); i++)
  {
    Serial.print(pgm_read_byte(referenceInfo + i));
  }
  Serial.println();
}

void loop()
{
  char incomingData[30];
  boolean dataAvailable = false;
  while (Serial.available() > 0)
  {    
    for (int i = 0; i < sizeof(incomingData); i++)
    {
      if (Serial.peek() > 0)
      {
        incomingData[i] = Serial.read();
      }
      else
      {
        incomingData[i] = 0;
      }
      delay(5);
    }
    dataAvailable = true;
  }
  if (dataAvailable)
  {
    Serial.println(incomingData);
    dataAvailable = false;
    if (incomingData[1] == ' ' || incomingData[1] == 0)
    {
      switch (incomingData[0])
      {
        case 'x':
        {
          int argument = atoi(incomingData + 2);
          if (argument > 0 && argument != currentTransmitTime)
          {
            Serial.print("Transmit time was ");
            Serial.print(currentTransmitTime);
            Serial.print(" ms but is now ");
            Serial.print(argument);
            Serial.println(" ms.");
            currentTransmitTime = argument;
          }
          else
          {
            Serial.print("Transmit time is currently ");
            Serial.print(currentTransmitTime);
            Serial.println(" ms.");
          }
          break;
        }
        
        case 't':
        {
          String argument = String(incomingData + 2);
          Serial.print("Transmitting command ");
          Serial.print(lastCommand, HEX);
          Serial.print(" for ");
          Serial.print(currentTransmitTime);
          Serial.println(" ms.");
          break;
        }
        
        case 'h':
        {
          int argument = atoi(incomingData + 2);
          break;
        }
        
        case 'e':
        {
          
          break;
        }
        
        case 'i':
        {
          
          break;
        }
        case 'm':
        {
          
          break;
        }
        case 'a':
        {
          
          break;
        }
        case 's':
        {
          
          break;
        }
        case 'f':
        {
          
          break;
        }
        case 'c':
        {
          
          break;
        }
        case '?':
        {
          help();
          break;
        }
        default:
        {
          Serial.println("Not a valid command");
          break;
        }
      }
    }
    else
    {
      Serial.println("Not a valid command");
    }
    delay(100);
  }
}

It’s running on the Diecimila and Uno just fine. I do have another concern, however. If I don’t add the delay here:

while (Serial.available() > 0)
  {    
    for (int i = 0; i < sizeof(incomingData); i++)
    {
      if (Serial.peek() > 0)
      {
        incomingData[i] = Serial.read();
      }
      else
      {
        incomingData[i] = 0;
      }
      delay(5); //<-------This one
    }
    dataAvailable = true;
  }

the program will tend to treat the first character of the input as a command, followed by all of the other characters as a separate command. Adding the delay solved the problem…but why?

Thanks!

I have succeeded in wiping the bootloader on two Diecimilas

If you have a sketch that reliably wipes out the bootloader, I'd love to see a copy. That OUGHT to be impossible (apparently it's not; I think even I've done it, but I don't think it has ever been explained HOW code manages to break the bootloader. I'd like to track that down...)

His fuse bits might be wrong, thus allowing access to program memory.

antiquekid3: I'm currently working on a ground station transmitter controller for a satellite that I helped build over the summer. It's getting ready to launch October 27th, so I'm in a bit of a hurry to get the final touches on the software end of things to control the transmitter. Here's what I've run into tonight:

You know, the AVR chips and the Arduino platform are fantastic little widgets that allow you to add a bit of intelligence to all kinds of things (anything from light switches to costume hats to beer dispensers :-)

However, they are not "the" solution for all possible embedded systems.

When it comes to "satellite controllers" with command-line interfaces, I'd suggest using something more like a WRAP/ALIX box or a Cortex M3 based board or similar. Examples:

ARM based boards: http://search.digikey.com/us/en/products/EKT-LM3S1968/726-1130-ND/1794060 (the board + software is $59) http://www.ti.com/lit/ug/spmu037a/spmu037a.pdf http://www.ti.com/lit/ds/spms037f/spms037f.pdf

Do you want a nice, 3.5" touch screen with your controller, and gobs of RAM as well? (Think "smartphone innards") http://www.mini-box.com/mini2440v2 ($109 -- 128+2 MB flash, 64 MB RAM)

ALIX box (linux based): http://pcengines.ch/alix.htm http://pcengines.ch/order1.php?c=4 (ALIX 3d2 is $99)

What you don't get with those systems is a support community, though. If that's what you want, how about a $199 system that you can develop on, AND surf the web on, AND end up sticking in whatever enclosure you'll end up using? It can provide all the buttons and user interface you want. Buy a netbook! http://www.newegg.com/Product/Product.aspx?Item=N82E16834246159 http://www.amazon.com/Asus-Netbook-1018P-BBK804-Processor-Aluminum/dp/B004VD2RYE

The closer to a "real" computer you get, the less you get "general purpose" I/O that you can hook motor drivers or whatever into. If the reason you're using an Arduino is that you need to hook up various I/O, then perhaps the right choice is a computer interface on a bigger box, and the Arduino is simply an I/O driver controlled by that bigger box using some simple serial protocol? You can have a user-friendly UI on the bigger box, because the serial protocol can be simple and only needs to worry about the needs of the machines.

Adding the delay solved the problem...but why?

Think of how slowly serial data gets transferred. About like I type.

If you read what I hav e typed in a g iven per iod of ti me, you won' t get a comp lete senten ce.

If you read slow enough, you'll get the complete sentence.

If you wait until I type an end-of-sentence marker before reading (period, exclamation point, or question mark), you'll get a complete sentence.

What you are doing now is reading slow enough. That is a band-aid solution, though. Change the baud rate, or give the sender or receiver enough other work to do, and the actual transfer rate will drop, and you will again miss getting a complete packet.

http://jhaskellsblog.blogspot.com/2011/05/serial-comm-fundamentals-on-arduino.html

His fuse bits might be wrong, thus allowing access to program memory.

It's unlikely that the fuse bits are wrong. Even if they are, it would be unlikely that they would be wrong in a way that a sketch could write the bootloader section. And even if THAT were true, there are odd protection procedures for writing pgmspace that are supposed to prevent it from happening unless you really mean it (Of the form "output this code to IO register X, and then do Y within 4 machine cycles.")

Which is why I'm really curious!

Hmm, OK then. Well I "wiped the bootloader" recently in a different way, so perhaps we need more detail about this "wipe the bootloader" process.

I received a Pololu ISP (USB) programmer recently. I tested it by fiddling around with a couple of configuration files (boards.txt, programmers.txt) so it was recognized. Then the sketch uploaded OK. However afterwards when I went back to the "normal" bootloader programming (via the USB port on the Uno) it kept timing out. I eventually worked out that using this other programmer had "wiped the bootloader". So the sketch didn't, it was my process of using another bootloader. Why, I'm not sure. I didn't really care at the time. Possibly uploading via the ICSP port caused the fuse bits to be set in such a way that the bootloader was not entered. Maybe the upload simply wiped it. I could re-test if this was thought helpful.

[quote author=Nick Gammon link=topic=75599.msg572036#msg572036 date=1318883136] programmers.txt) so it was recognized. Then the sketch uploaded OK. However afterwards when I went back to the "normal" bootloader programming (via the USB port on the Uno) it kept timing out. I eventually worked out that using this other programmer had "wiped the bootloader" [/quote]

My guess is that your uploading the sketch actually replaced the bootloader. If the available memory is 32 kB, the normal upload process protects 2 kB for the bootloader, and gives you 30 kB for the sketch. However, with a bit-banger programmer, I imagine that all 32 kB is available for your sketch, and it ignores the bootloader and overwrites with your sketch. If you want to go back to serial-based boot loader, you'll have to re-program the boot loader using the bit-banger again. (I generically use "bit-banger" for any programmer that doesn't go through the 2 kB bootloader serial interface, but instead uses lower-level methods)

I agree. Which is why I suggest that perhaps the sketch didn't "wipe" the bootloader, it was uploading the sketch that did it.

it was uploading the sketch that did it.

The normal ISP programming procedure for uploading to an AVR (ie without a bootloader) starts with a "chip erase" sequence, which wipes out the bootloader memory as well as the rest of flash and the protection bits (the only thing it doesn't change s the fuse bytes.) This is a required part of ISP (or parallel programming); unlike the bootloader, ISP cannot erase a page of eeprom at a time before reprogramming it.