Problem converting HTML number codes

Projects Programming
1

Hi, I have a script that takes the last message posted on a Tumblr account using an Uno/ethernet shield and displays it on an LCD. I use a String to store the last post, as I don't think char arrays have the functionality I need. It works pretty well, but in the text my program is getting, some symbols are getting converted to html codes. Although there may be more, here are all the conversions I've seen:

" - “ or ”
' - ‘ or ’
& - &
— - —
< - <

  • >

I started to put lines of code in my script to replace these html codes with their corresponding character. Here are some examples:

text.replace("&#8220;","\""); 
text.replace("&#8221;","\"");
text.replace("&#8216;","'");
text.replace("&#8217;","'");

This works, but if I add a lot of these replace statements, something goes wrong: My text String turns into an empty String. I think this has something to do with memory use. (If someone knows why this happens, I would really like to hear it). I'm already using other replace statements to remove the "

", "

", "
", "
", etc.. occurrences in my text String, so I'm unable to use all the other replace statements I need without my text String going empty on me.

Basically, I'm wondering if any of you know of (1) a library I could use that fixes HTML code, or (2) a way to replace the HTML codes without wasting my text String memory.

Here's info about my implementation if that helps:
I initialize my text String globally:

String text = "";

I reserve memory for it in setup() (raising this number higher than 300 doesn't seem to help my problem btw):

text.reserve(300);

In my loop, I clear the String whenever I reach the beginning tag of the text I need to parse (this only happens once per cycle):

text = "";

Then I add chars onto my text String until I reach the end tag of the text I'm parsing:

text.concat(inChar);

This leaves my text String about 200 to 250 characters long.
Then I trim off beginning/end whitespace and use replace statements:

text.trim();
text.replace("<p>","");
text.replace("&#8216;","'");

Then I find the newline locations in my text String which I use to print substrings of each line to an LCD.

Thanks for taking the time to read all of this!

2

If someone knows why this happens, I would really like to hear it

http://lmgtfy.com?q=arduino+string+memory+problems

Instead of replacing in the same string, which will really muller your memory, you should be building up a new string a character at a time, and when you hit & you parse the data up to a ; and add the new text. You could store the key/value pairs in an couple of arrays in progmem and iterate through them to find the matches. Oh, and that string should be a string not a String (i.e., a char * not a String object).

3

Thanks for the reply. I was aware that Strings had memory problems of some sort. I wasn't sure if the replace() issues I was having were really related to that though. I thought I could possibly get away with using Strings, but I think it's time to make the switch to char arrays.

I just found out about the string.h library so it looks like the transition won't be as hard as I thought.