Out of Memory on Mega, Have an extra uno, Combine?

Here is the time stamp string

TS = String(year()) + "/" + String(month()) + "/" + String(day()) + "," + String(hour()) + ":" + String(minute()) + ":" + String(second()); That allows me to store the data in an excel sheet that looks like 1/29/2015 in the first cell, and 10:45:00 Pm in the second cell. Then in the third, forth, and fifth cell other data from sensors are stored.

So I should keep each as a char to save memory?

I'm very interested in figuring out how to make two arduinos share the load.

A simple example for sending time and data to CSV is here. By virtue of the type of file, the commas do the vital assignment to the cells. No string assembly is required, and no char either. The method for sending data to SD is essentially the same as that to serial and display.

 k=k+1;  

  if (k>9 )
  {  
           myFile = SD.open(filename, FILE_WRITE);//<<<<<<<<<<<<< OPEN
  myFile.print(hour);
  myFile.print(":");
  myFile.print(minute);
  myFile.print(":");
  myFile.print(second);
  myFile.print(",");

  myFile.print(InTemp);
  myFile.print(",");
  myFile.print(OutTemp);
  myFile.print(",");
  myFile.print(DrainTemp);
  myFile.print(",");
  myFile.print(diff);
  myFile.println();
       myFile.close();//>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>CLOSE     
      k=0;
  }

The only reason why you could possibly be interested in using two Arduinos is that you refuse to do a proper job using one and, in the unlikely event that you successfully bring a Uno into the game, the best memory increase you can expect is about 10%, which was probably never worth the effort. I rather suspect that, if you ever run out of memory with a Mega, you probably need to be using something other than an Arduino.

Thomas499:
Excellent, so in theory if I get the due, I can use the due for the heavy lifting, and the mega for the shields and other fun stuff. Would that be through wire? Is there an example out there that shows how to share the load?

I wonder if a Raspberry PI (or a cheap laptop) would be a better companion for your Mega? Just use the Mega for the stuff an Arduino is good at.

...R

Thomas499:
I have an extra uno laying around, am I'm curious if it's possible to take the memory from the uno, and add it to the Mega somehow?

Unfortunately not.

I must say I have no experience with it, but you can... apparently add ram to an mega2560 using a quadram - shield/module
It should give you 512KB of RAM and a smaller one, mega-ram with 128KB is available as well.

The only reason why you could possibly be interested in using two Arduinos is that you refuse to do a proper job using one and, in the unlikely event that you successfully bring a Uno into the game, the best memory increase you can expect is about 10%, which was probably never worth the effort. I rather suspect that, if you ever run out of memory with a Mega, you probably need to be using something other than an Arduino.

Let me explain in more detail. I have plenty of memory left of the mega, I am just short on the sram memory. I've used, the F()macro as much as I can, but when it comes to using the aruino as a server, using the client.print(F()); results in everything not being printed because it takes to long to pull from whatever memory bank F uses. As a result, if you wanted a full page to show up, it stops at half page. So you can't use the F() macro, the same appears to apply for writing to an sd card.

So I was thinking trying it out with the uno, because I have an extra laying around, and if I can get a simple one boolean command to work i'll purchase the due, which I am confident will be more than adequate, So I can use the mega for the shields, and boolean calculations, then send the boolean results to the due, which the due will then work as the server. If I can get the due to work with a wi-fi shield that is. The due should add an additional noticeable amount of memory. Of course, I have no idea how to go by making them talk to each other to share the load.

What I really need is a super computer server, like the one Nick Gammon uses to host his video game. My resources are limited though, and I'd like to figure out how to do everything on an arduino before trying to learn how to program a sever though. With any luck i'll try to get my hands on one in the next two years.

I wonder if a Raspberry PI (or a cheap laptop) would be a better companion for your Mega? Just use the Mega for the stuff an Arduino is good at.

I've looked into getting a raspberry pi a little. I'm still getting use to arduino technology and the pi appears to be much more complicated. Do you ever use the pi? I'm curious how I would go by using a laptop to share the load with the arduino? That would be all the memory and processing power I would need.

I must say I have no experience with it, but you can... apparently add ram to an mega2560 using a quadram - shield/module
It should give you 512KB of RAM and a smaller one, mega-ram with 128KB is available as well.

This seems like a possible fix as well. Has anyone used a shield like this before?

Are you certain you can't combine SRAM variables, or break things into subroutines and allocate the SRAM in a function? I have some pretty complex programs, and I don't use nearly all the SRAM available in my Mega 2560.

If you are willing to post your code, maybe someone will be able to help you use your SRAM more efficiently.

I don't think the variables are the issue, i think it's what I want printed.

Say for example I wanted "Grandma fell down the steps" printed on on the sd card, and on the webserver. The two booleans would be Steps = HIGH; and Grandma = HIGH; Those are the variables right? The sentence that I want printed seems to take up the memory, and I can't use the F()Macro because then it wouldn't print everything. I could probably include a lot of delays so that everything would print, but I really really don't like using delays.

The sentence that I want printed seems to take up the memory, and I can't use the F()Macro because then it wouldn't print everything.

Well, yes, it has to take up memory, there being no other place for it to be, but I don't understand your comment about it not printing everything.

Please post your code (the whole thing) if you want help on how to reduce it's sram usage. Otherwise we're all blindfolded in a sea of unknown code trying to play marco-polo with sram use

Re: memory expanders: I had no idea that the Mega could get it's memory expanded like that; that's really neat - it looks like it's got direct hardware support baked into the ATMEGA2560.

I don't think the variables are the issue, i think it's what I want printed.

Then I guess you are on your own. But I will give you a word of advice. Go through your global variables and see if there aren't some arrays you could combine or allocate in functions.

Just as an example, I use one character array to print pretty much everything in my code. Normally I refer to this array as tBuffer[] or tBuf[] in each function, and print everything into it with sprintf or strcpy/strcat as much as practical. That way when the function exits (returns), that array memory is released.

Look at this example. It is one of my most complex sketches on the playground.
http://playground.arduino.cc/Code/WebServerST
Note how the arrays are allocated in the checkServer function. When that function exits, all that memory is released for use in the myStuff function, even though the myStuff function doesn't do much of anything in that example sketch. It is there if you need it.

Thomas499:
I've looked into getting a raspberry pi a little. I'm still getting use to arduino technology and the pi appears to be much more complicated. Do you ever use the pi? I'm curious how I would go by using a laptop to share the load with the arduino? That would be all the memory and processing power I would need.

You have not actually told us what your project is intended to do. The advice would be a lot more relevant if you tell us.

I assumed from Reply #2 that part of the project involves collecting data from sensors and another part of it involves sending that data somewhere to add to a database, or to display for a user. A cheap laptop would be ideal for the storage or presentation as it has no practical limits on the size of Strings or of data.

A RaspberryPI is just like a laptop without a screen, keyboard or power supply and on a very small circuit board. If you have to buy a screen, keyboard and power supply a small used laptop might be much the same price. The RaspberryPI uses the Linux operating system.

...R

I assumed from Reply #2 that part of the project involves collecting data from sensors and another part of it involves sending that data somewhere to add to a database, or to display for a user. A cheap laptop would be ideal for the storage or presentation as it has no practical limits on the size of Strings or of data.

A RaspberryPI is just like a laptop without a screen, keyboard or power supply and on a very small circuit board. If you have to buy a screen, keyboard and power supply a small used laptop might be much the same price. The RaspberryPI uses the Linux operating system.

Can anyone show me an example code that uses a lap top, or raspberry pi to do the storage or presentation?

Is there a way to tell the difference between needing more processing power, and needing more sram memory?

Thomas499:
Is there a way to tell the difference between needing more processing power, and needing more sram memory?

Pop this function somewhere in your sketch

int get_free_memory()
{
 extern int __heap_start, *__brkval;
 int v;
 return (int) &v - (__brkval == 0 ? (int) &__heap_start : (int) __brkval);
}

Then at some strategic points in your sketch

Serial.print( F("Free memory ") );
Serial.println( get_free_memory() );

Thomas499:
Can anyone show me an example code that uses a lap top, or raspberry pi to do the storage or presentation?

For that you probably need to try a Raspi or laptop forum, but I doubt that you will get much help since you are so secretive about what you want to do and the Arduino code you might already have.

I also assume from Reply #2 that part of the project involves collecting data from sensors and another part of it involves sending that data somewhere to add to a database, or to display for a user. This sounds pretty ordinary and is quite likely something done easily on an Arduino, just like thousands of others routinely do, but I guess nobody will ever know because you aren't telling - and there really isn't much point in guessing.

As for reply #17 you may find there is no need for more processing power, or more sram, you just need some sensible code.

Let me explain in more detail. I have plenty of memory left of the mega, I am just short on the sram memory. I've used, the F()macro as much as I can, but when it comes to using the aruino as a server, using the client.print(F()); results in everything not being printed because it takes to long to pull from whatever memory bank F uses. As a result, if you wanted a full page to show up, it stops at half page. So you can't use the F() macro, the same appears to apply for writing to an sd card.

Maybe you should explain better as to what you are doing. If it is all static text, or what ever, and your server is a web server, then you should be able to keep and upload the static material from an SD disk.

Thomas499:
Can anyone show me an example code that uses a lap top, or raspberry pi to do the storage or presentation?

This Python demo should get you started. This Python GUI demo may also be of interest.

...R

Maybe you should explain better as to what you are doing. If it is all static text, or what ever, and your server is a web server, then you should be able to keep and upload the static material from an SD disk.

That is pretty much what I want to do, but I want to keep the master information stored on a sd card(like peoples names and such), and use that to write to the sd card when certain buttons are pressed. The problem is, I can only have one file on the sd card open at a time so It's kind of a conflict of interest.

DrAzzy:
Please post your code (the whole thing) if you want help on how to reduce it's sram usage. Otherwise we're all blindfolded in a sea of unknown code trying to play marco-polo with sram use

Re: memory expanders: I had no idea that the Mega could get it's memory expanded like that; that's really neat - it looks like it's got direct hardware support baked into the ATMEGA2560.

Yeah, that AVR family allows external RAM to be addressed as internal, IIRC with 1 extra cycle.

I got a Rugged Circuits QuadRam shield for my MEGA since it was under 30 bucks shipped. You end up with the internal 8K devoted to stack use and 8 56K (64K - the internal 8K in the shared address space) banks for heap.

And still I wouldn't use C++ String objects on it. They waste cycles and RAM and shotgun the heap.

As per earlier advice: don't build up long strings just to print. That's wasteful. Serial data is passed a char or byte at a time and even that is passed one bit at a time. Send the characters as you have them.

if you can print very long text from F() macros then cut it into shorter pieces and print those one after the other. Did you even think to try?

I get this feeling that you labor under some false impressions about computers and code that come from what you have seen and maybe even been taught. If that is the case then I feel sorry for you and want you to know and have it better. It's really simpler than you seem to think or at least what you code for.

What you need to learn should be easier to pick up through small sketches. It's too easy to get lost in a pile of code that deals with many other issues. Work on some tutorials and then go back to your project with wiser eyes. Do that a few times before you work on that code since at some point you should rearrange most of it while keeping the bits you really need. The trail may be longer but you'll end up in a better place.

Please post your code (the whole thing) if you want help on how to reduce it's sram usage. Otherwise we're all blindfolded in a sea of unknown code trying to play marco-polo with sram use

See attachment. Using method 1 My Free Ram was 6608, using Method2 my Free Ram was 6432 and using Method 3 my Free Ram was 6424. Now that's only loosing 200, which doesn't sound bad, but when you take the way I was coding, and multiply that times a sketch that has 20 times that amount of variables, I ran out of Free Ram.

Using method 1 gets around the Free Ram issue, but it takes so much code, to be able to use the information in a Server code, and save the information to a sd card, and print it in the serial monitor for debugging using many many if statements each time, it's way to much code. If anyone can offer a solution that i'm not seeing, it would be greatly appreciated!

You end up with the internal 8K devoted to stack use and 8 56K (64K - the internal 8K in the shared address space) banks for heap.

I don't understand the concept of heap, but are you saying this basically take the 8k SRAM that I currently have and safely modify it to 56K? If so, that might do the trick.

As per earlier advice: don't build up long strings just to print. That's wasteful. Serial data is passed a char or byte at a time and even that is passed one bit at a time. Send the characters as you have them.

I don't understand how else to do it without using a billion if statements throughout the code. The simpler the code, and less lines, the better. At this rate I would use more lines of if statements, than the rest of the code combined. I tried putting it in an char instead of a string like this

if (timeZoneSD==1)
      { 
        TempNC = 0;
        while (MSChars > TempNC)
        { 
          char c = TZ1[TempNC];
          quickreferenceTZ[TempNC] = c;  
          TempNC++;      
          if (TempNC>MSChars)
          { 
            TempNC = MSChars - 1;
          }
        }
      }

but that didn't seem to help the SRAM issue much.

if you can print very long text from F() macros then cut it into shorter pieces and print those one after the other. Did you even think to try?

Are you saying you can use a F()macro and parse from that, instead of parsing from a string?

I get this feeling that you labor under some false impressions about computers and code that come from what you have seen and maybe even been taught. If that is the case then I feel sorry for you and want you to know and have it better.

That actually means a lot, and is very accurate. Thank you.

SRAM_Help_Needed.ino (8.37 KB)